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(54) Motion picture information displaying method and apparatus 

(57) An Image information displaying method and 
apparatus Ibr the edition of a motion picture are dis- 
closed in which representative image (1030-1033) rep- 
resenting a series of frame images forming the motion 
picture are displayed on a screen of a display device 
(3080) and a hierarchical structure (1210. 1220. 1230. 
1240) based on a plurality of representative images is 
generated on the saeen through an operation using a 
GUI to perform the edition of the motion picture. A plu- 
rality of frame images (321-330) included in the repre- 
sentative image are subjected to an image detection 
processing for detecting frame images having a prede- 
termined image recorded thereon. Information (614. 
616) indicating the detected frame image having the 
predetermined image recorded thereon is displayed as 
a detection result displaying window (600) on the 
screen. Information (250-252) concerning frame images 
having not been subjected to the image detection 
processing is displayed on the detection result display- 
ing window. 
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Description 

BACKGROUND OF THE INVENTION 

[0001] The present invention relates to an image 
information displaying method and apparatus in which a 
motion picture is edited by operating icons or screen 
menus on a display screen by means of a QUI (Graphi- 
cal User Interface). 

[0002] In recent years, the enhancement of speed 
of computers and the improvement in functionality 
thereof have brought about the rapid progress of the 
variegation and improvement in functionality of soft- 
ware. The improvement in functionality of a production 
equipment for produdng or arranging a TV broadcast 
program or video program has been advanced by use of 
a computer and an image editing program. 
[0003] Particularly, in the recent program arrange- 
ment, the edition of image information using a motion 
picture editing apparatus with a random access infor- 
mation storage device (for example, a hard disk which is 
easy of handling and preservation and is low in cost) is 
made in lieu of an image editing method in which a 
video tape recorder is used to edit an image by repeat- 
ing the quick feed and rewind depending a tape counter. 
The use of such a motion picture editing apparatus in 
combination with a computer and software improved in 
functionality and variegated has made the progress of 
more improvement in functionality of the nrwtion picture 
editing apparatus. Here, the image information is any 
information which concerns an image and may include 
voice. 

[0004] Rg. 4 shows an example of display on a 
screen used for an edition processing in a motion pic- 
ture editing apparatus for editing a motion picture by use 
of a computer. With the use of a motion picture editing 
apparatus provided witti a required edition processing 
software in a memory device, image information of a 
motion picture to be edited can be managed in a man- 
ner stored in the memory device Reduced versions of 
cut images or scene images of a motion picture stored 
in the memory device are displayed as moving icons (M- 
icons) on a monitor screen to form a tree structure hav- 
ing cut images or scene images as its unit, ttiereby mak- 
ing it possible to perform an editing work for the motion 
picture. In the screen shown in Fig. 4. reference 
numeral 1000 denotes an editing window, numeral 1010 
an editing area, numeral 1020 a material area, numer- 
als 1030. 1031. 1032. 1033. — . 103K. 103M. 103N. — 
moving icons of image materials or sources displayed in 
the material area 1020 (where K. M and N are natural 
numbers and K = N-2 and M = N-1 are satisfied), and 
numeral 1040 a tree structure which is formed by mov- 
ing icons and represents the contents of edition of an 
edited motion picture. 

[0005] The cut images, scene images, moving 
icons (M-icons) and so fortii will now be described. 
[0006] A cut image 1210 is a set of frame images 



obtained through one photographing by a TV camera 
from its start to its end, and a representative image rep- 
resenting a series of frame images. 
[0007] A scene image 1220 is a set of plural cut 
5 images 1210 obtained for a predetermined object of 
photographing. 

[0008] A plurality of scene images 1220 may be 
gathered to provide a block 1230. A plurality of blocks 
1 230 can be gathered on the t>asis of a specified theme 
10 of photographing to provk:le or display an edited motion 
picture (or program) 1240 of the specified photograph- 
ing theme. 

[0009] Thus, a motion picture to be edited can be 
arranged as image data with a hierarchical structure (or 

15 tree structure) which may be formed by several cut 
images 1210 selected from among a plurality of cut 
images 1030. 1031. 1032. 1033,—, 103K. 103M. 103N. 
— as image materials in compliance with a specified 
theme of photographing, scene images 1220 each 

20 including the collection of a plurality of cut images 1210. 
t>locks 1230 each including a group of selected scene 
images 1220. and a motion picture 1240 including the 
collection of blocks 1230 and providing a program with a 
titie. 

25 [0010] This tree structure is ordered and applied 
with data of a series of frame numbers for a motion pic- 
ture, frame numbers of images at detected cut change 
points, cut image numbers serially applied to cut 
images, and sequence numbers representing the 

30 sequence of plural cut images each forming a scene 
image. This data or hierarchical structure information is 
included in control information used in the motion pic- 
ture editing apparatus. 

[0011] The M-icons are GUI (Graphical User Inter- 

35 face) parts each having a one-to-one con-espondence 
to a cut image, that is, a representative image repre- 
senting a series of frame images obtained tfirough one 
photographing from its start to its end. 
[0012] The M-icon is an image for tfie purpose of 

40 display which has a size reduced at a required ratio. The 
M-icons are displayed in tiie case where scene images 
or the whole or a part of a motion picture formed by a 
plurality of frame images are to be viewed promptly, as 
required. Otherwise, the M-icons may be displayed as 

45 representative ones of cut images. 

[0013] The monitor screen provkies a GUI screen 
for displaying edition information of cut images, scene 
images and a tree structure representative of the state 
of edition which is used for the edition of a motion pic- 

50 ture. An editor uses an input device (for example, a 
mouse or keyboard) to select this edition information 
displayed on tiie monitor screen, thereby making it pos- 
sible to give an insti-uction to the motion picture editing 
apparatus. 

55 [001 4] Thus, with tiie construction in which a motion 
pfoture to be edited is stored and managed as a tree 
structure, ttie motion picture editing apparatus is ena- 
tJled to arrange a program and to perform an editing 
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work with cut images or scene images taken as units. 
[0015] The screen of the motion picture editing 
apparatus shown in Fig. 4 can be used, as a specific 
exanple. in the case where image materials 1030-1033 
of an existing news program are used as materials of 
edition to prepare another news program (e.g. a digest 
versk)n) newly. 

[0016] In the display screen shown in Fig. 4. refer- 
ence numeral 1050 denotes a group of edition function 
buttons for selecting various control functions used in an 
editing worK numeral 1051 a character (or appearing 
person) detecting button, numeral 1052 a tree structure 
forming button, numeral 1070 a slider for sliding the dis- 
play contents of the editing area 1010 in a lateral direc- 
tion, numeral 1080 a slider for sliding the display 
contents of the editing area 1010 in a longitudinal direc- 
tion, numeral 1090 a slider for sliding the display con- 
tents of the material area 1020 in a longitudinal 
direction, numeral 1210 M-icons representative of cut 
images registered in the editing area 1010. numeral 
1220 an M-icon representative of a scene image formed 
by one or plural cut images, rujmeral 1230 an M-icon 
formed one or more scene images, numeral 1240 an M- 
icon representative of a motion picture (or program) 
edited in compliance with a given theme, and numeral 
810 a mouse pointer displayed on the GUI screen when 
a GUI operation is performed by a pointing device such 
as a mouse. Though the M-icon has thereon the corre- 
sponding reduced image for the purpose of display, the 
illustration or depiction of this reduced image will be 
omitted for avoiding the complexity, so long as there is 
no special necessity. 

[001 7J In Fig. 4. the tree structure 1040 formed by 
the M-icons in the editing area 1010 is such that one or 
plural M-icons 1210 on a first hierarchical layer repre- 
senting cut images selected for edition are collected to 
form an M-icon 1220 representative of a scene image 
as a second hierarchical layer on the first hierarchical 
layer, one or plural M-icons 1220 representing the 
scene images are collected to form an M-icon 1 230 rep- 
resentative of the collection of specified scene images 
as a third hierarchical layer on the second hierarchical 
layer, and a plurality of M-icons 1230 each representing 
the collection of specified scene images are finally col- 
lected to form an M-icon 1240 representing a program 
edited in conpliance witii a theme. 
[001 8] The number of hierarchical layers in the tree 
structure 1040 may be arbitrary. It is of course that an 
upper hierarchical layer can further be provided. Since 
the size of the window of the editing area 1010 is lim- 
ited, there may be the case where it is not possit^le to 
display the whole of the tree structure 1040 within the 
editing area 1010. In order to see an undisplayed por- 
tion, the slider 1070 or 1080 is operated by use of ttie 
input device to move the display contents of the editing 
area 1010 in a required lateral or longitudinal direction, 
thereby making it possible to view the whole of the tree 
structure 1040. 



[0019] In the material area 1020. different cut 
images obtained by detecting cut change points for a 
series of motion picture materials and dividing tiiem at 
the cut change points are temporarily located as image 
5 materia! M-icons 1030, 1031. 1032, 1033, — . 103K. 
103M. 1 03N. —which are displayed so that it is possible 
to take a look at them. 

[0020] At a stage in which the editing work pro- 
ceeds to the invocation of images so that M-icons 1030, 

10 1031 , 1032, 1033. — . 103K. 103M. 103N. — are gener- 
ated and displayed, the M-icons 1030. 1031. 1032. 
1033, — . 103K. 1Q3M. 103N. — are displayed in ttie 
material area 1010 at a time serial sequence in the left 
to right direction and the up to down direction. 

15 [0021] Also for the M-icons 1030. 1031. 1032. 
1033, — . 103K, 103M, 103N, — in the material area 
1020, there may be the case where the number of the 
M-icons is large so ttiat they cannot be displayed in the 
window of the material area 1020 at once, as in the case 

20 of the editing area 1010. In that case, the slider 1090 
can be operated by use of the input device to move the 
display contents of tiie material area 1020 in the longi- 
tudinal direction, thereby making it possible to view all 
the M-icons. 

25 [0022] In order to perform the edition. M-icons 
required for edition are selected from the material area 
1020 and ttie tree structure forming button 1052 in the 
function button group 1050 is then turned on. Thereby, 
tiie selected M-icon 1030 is moved or copied into the 

30 editing area 1010. Thus, tiie tree structure 1040 is 
formed to make progress on tiie edition. 
[0023] Fig. 4 shows a state in which tiie progress of 
tiie edition to a certain extent is attained so that the tree 
structure 1040 has already been formed in tiie editing 

35 area 1010 and tiie edition is tiien continued to further 
select desired M-icons from among the M-icons 1030. 
1031. 1032. 1033. — . 103K. 103M. 103N. — in tiie 
material area 1020. 

[0024] The M-icon displayed in tiie material area 

40 1020 or the editing area 1010 is shown as one card in 
ttie illusti-ation of the figure. However, ttiis is a simplified 
form. On the actual monitor screen, each M-icon is dis- 
played in a form having the thickness of superimposition 
of frame images possessed by that M-icon. 

45 [0025] Though ttie edition of the motion picture is 
advanced, there is less the case where image materials 
actually required for the arrangement of a program are 
all image information stored in ttie information storage 
device (cut images or a series of motion pictures read 

so as a file of one motion picture). 

[0026] Namely, tiiere may not be tiie case where all 
the plurality of M-icons 1030, 1031. 1032. 1033. — . 
103K. 103M. 103N. — displayed in ttie material area 
1020 become necessary Only image materials assod- 

55 ated with some M-icons selected in accordance witii an 
object of edition or a theme of photographing are used 
for a program. 

[0027] Also, since it is general tiiat a plurality of 



3 



BNSDOCIO: <EP 103371BA2J_> 



EP1033 718A2 



6 



scenes are induded in the image materials, a search is 
made so that M-icons required for edition are selected 
from among a multiplicity of M-icons 1030, 1031, 1032, 
1033. 103K. 103M. 103N. - included In image 
materials or image information read from the informa- 
tion storage device. 

[0028] The operation of the motion picture editing 
apparatus using the screen shown in Fig. 4 and the 
metfiod of performing the edition of a motion picture 
while forming a tree structure by use of M-icons have 
been disclosed by. for example, JP-A-l 0-51 733. 
[0029] Also. JP-A-2-184181 has disclosed an 
example of means for detecting change points for cut 
Images, that is. a technique of analyzing the illumi- 
nance, sound volume, color tone or the like of a motion 
picture to detect a portion with a remarkable change as 
a change ix>int for a cut Image. 

[0030] In order to perform the edition of a motion 
picture as mentioned above, it is required for an editor 
that an M-icon representing a cut necessary for the pro- 
gram should be selected from among the plurality of M- 
icons 1030. 1031. 1032. 1033. — . 103K. 103M. 103N. - 
-<lisplayed in the material area 1020. 
[0031] One function supporting the search for the 
necessary M-icon is a character detecting function. The 
character detecting function detects an object such as a 
specified character or person from image materials or 
plural cuts or scenes. Here, this specified character, that 
is object is assumed as "X". 

[0032] With the character detecting functfon. an 
operator selects and inputs the combination of features 
possessed by tfie object "X" so tfiat a search for a frame 
conformal)le to those features and its pixel region is 
made from image materials or plural cuts or scenes 
(that is. images in a section subjected to detection) by 
use of an inrtage recognition technique. 
[0033] JP-A-6-223179 has disclosed a method in 
which a cut containing a specified object therein is 
detected from a motion picture on the basis of the fea- 
tures of that character. 

[0034] Rrst. tfie operator designates the combina- 
tion of features (for example, hair color, skin color and 
dress color) of a specified person of an object "X" in a 
series of selected M-icons (that is, a section subjected 
to detection where tfie character "X* may appear). 
Thereby, the nation picture editing apparatus detects 
an image region conformable to tiie designated cont>i- 
nation of colors through an Image processing so tiiat 
information of tiie conforming image and its frame is dis- 
played on the GUI screen. 

[0035] When the frame containing the character "X" 
therein is detected, the editor confirms a cut which 
includes that frame. If this cut is appropriate as a part of 
the program material, the cut is selected and is inserted 
into the hierarchical structure 1040 of the editing area i 
1010 shown in Fig. 4. 

[0036] In tiie conventional motion picture editing 
apparatus, a frame containing a specified image is 



detected from a motion picture Ixit tiiere is not provkled 
a function of displaying information of the result of 
detection in a form which is conprehenstole for an edi- 
tor to use as infonnation for edition. Also, the conven- 
5 tional motion picture editing apparatus is not capatrfe of 
displaying a motion picture range having already been 
subjected to a processing for detection of a frame con- 
taining a specified character therein and a range having 
not yet been sulDjected to the detection processing so 
10 that they are displayed on a screen in a distirxjtive man- 
ner. Further, tiiere is not provkled a function of make a 
correction or change for the result of detection. 

SUMMARY OF THE INVENTION 

15 

[0037] An object of the present invention is to pro- 
vide a method and apparatus in which a frame contain- 
ing a specified image therein is detected from a motion 
picture on the basis of features possessed by the spec- 

20 ified image and information off the result of detection Is 
displayed on a motion picture editing screen in a form 
comprehensible for an editor. Anotiier object of the 
present invention is to display a motion pfoture range 
having already been subjected to a processing for 

25 detection of a frame containing a specified character 
therein and a range having not yet been subjected to 
the detection processing so that they are displayed on a 
screen in a distinctive manner. A further object of the 
present invention is to provide an editing screen with a 

30 function of making a correction or change for tfie result 
of detection. The specified image to be detected in the 
mettiod or apparatus of tfie present invention is not lim- 
ited to a character or person and may include a physical 
solid other than a character, a scenery (or landscape) 

35 and so fortii. 

[0038] In an image information displaying method 
for the edition of a motion picture in one aspect of the 
present invention, representative image representing a 
series of frame images forming the nx)tion picture are 

40 displayed on a screen of a display device and a hierar- 
chical structure based on a plurality of representative 
images is generated on the screen through an operation 
using a GUI to peribrm tfie edition of tfie motion picture. 
According to the present invention, a plurality of frame 

45 images included in the representative image are sub- 
jected to an image detection processing for detecting 
frame images having a predetermined image recorded 
thereon. Information indicating the detected frame 
image having the predetermined image recorded ther- 
50 eon is displayed as a detection result displaying window 
on the screen. Information concerning frame images 
having not t)een subjected to the image detection 
processing is displayed on the detection result display- 
ing window. 

55 [0039] An image information displaying apparatus 
for the edition of a motion picture according to another 
aspect of the present invention comprises a display 
device having a screen for displaying representative 
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images representing a series of frame images forming 
the motion picture, a control device for controiling image 
information displayed on the screen of the display 
device, a storage device for storing the frame images, 
the image information and data for managing the frame 5 
images and the image information, and an input device 
for inputting data into the control device. The control 
device includes means with which a hierarchical struc- 
ture based on a plurality of representative images is 
generated on the screen of the display device on the 10 
basis of data inputted by the input device and in 
response to an operation using a GUI. means with 
which a plurality of frame images included in the repre- 
sentative image are subjected to an image detection 
processing for detecting a frame image having a prede- 15 
termined image recorded thereon, means with which 
information indicative of the detected frame image hav- 
ing the predetermined image recorded thereon is dis- 
played as a detection result displaying window on the 
screen, and means with which information concerning 20 
frame images having not been subjected to the image 
detection processing is displayed on the detection result 
displaying window. 

[0040] The image information displaying method for 
the edition of a motion picture according to the present 25 
invention can be stored in a recording medium as a pro- 
gram which can be read t>y a computer. The computer 
readat>le medium has computer readable program code 
means comprising means with which representative 
images representing a series of frame images forming 30 
the motion picture are displayed on a screen of a display 
device, means with which a hierarchical structure based 
on a plurality of representative images is generated and 
displayed on the screen of the display device on the 
basis of inputted instruction data, means with which a 35 
plurality of frame images included in the representative 
image are subjected to an image detection processing 
for detecting frame images having a predetermined 
image recorded thereon, means with which information 
indicative of the detected frame image having the pre- 4o 
determined image recorded thereon is displayed as a 
detection result displaying window on the screen, and 
means with which information concerning frame images 
having not been subjected to the image detection 
processing is displayed on the detection result display- 45 
ing window. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0041] so 

Rg. 1 is a diagram showing an embodiment of an 

IDX window in the present invention; 

Fig. 2 is a diagram an example of an ID window for 

explaining the detection of character; ss 

Fig. 3 is a block diagram showing an example of the 

construction of a motion picture editing/displaying 

apparatus; 



Fig. 4 is a diagram showing an example of display 
on an editing window; 

Fig. 5 is a diagram showing an example of an initial 
slate of an IDX window in the pres nt invention; 
Rg. 6 is a diagram showing an example of display 
of the result of image detection on an IDX window; 
Rg. 7 is a diagram showing an embodiment of an 
IDX window in the present invention; 
Rg. 8 is a diagram showing an example of an edit- 
ing window; 

Figs. 9A, 9B and 9C are views showing that the 
result of a detection of an object is displayed on a 
side face of an M-icon which represents a repre- 
sentative image in the form of a solid figure; 
Fig. 10 is a diagram showing an embodiment of an 
IDX window in the present invention; 
Rg. 1 1 is a diagram showing an errtbodiment of an 
IDX window in the present invention; 
Figs. 12A. 128 and 12C are views showing that the 
result of a detection of an object is displayed on a 
side face of an M-icon which represents a repre- 
sentative image in the form of a solid figure; 
Rg. 13 is a diagram showing an example of display 
on an ID window on which the confirmation of for 
the result of a detection of an object or a correction 
therefor is made; 

Rg. 14 is a diagram showing an example of display 
on an ID panel; 

Rg. 15 is a diagram showing an emkxxJiment of an 
editing window in the present invention; 
Fig. 16 is a diagram showing an embodiment of an 
ID panel in the present invention; 
Rg. 1 7 is a diagram showing an embodiment of an 
ID panel in the present invention; 
Rg. 18 is a diagram showing an entbocfiment of a 
monitor screen on which an editing window, an ID 
panel arKi an IDX panel are simultaneously dis- 
played; 

Fig. 19 is a diagram showing an embodiment of an 
IDX panel in the present invention; 
Fig. 20 is a diagram showing an editing window 
according to an emlxxJiment of the present inven- 
tion; 

Rg. 21 is a diagram showing an editing window 
according to an embodiment of the present inven- 
tion; 

Rg. 22 is a diagram showing multi-window display 
according to an emtxxjiiment of the present inven- 
tion; 

Rg. 23 is a diagram showing an embodiment of an 

IDX panel in the present invention; 

Rg. 24 is a diagram showing an ID panel according 

to an emtxxiiment of the present invention; 

Fig. 25 is a diagram showing an embodiment of a 

nxjiti-window screen on which an editing window. 

an ID panel and an IDX panel are simultaneously 

displayed; 

Rg. 26 is a diagram showing an embodiment in 
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which an editing window, an ID panel and an IDX 
panel are simultaneously displayed; 
Fig. 27 shows a flow chart showing an enixxJiment 
of a processing for making a connection for a detec- 
tion effectuating domain according to the present s 
invention; 

Rg. 28 is an embodiment of an ID panel in the 
present invention; 

Rg. 29 is a diagram showing an ID panel according 
to an embodiment; yo 
Rg. 30 shows a flow chart showing an entxxliment 
of a processing for interpolation change In selected 
frame according to the present invention; 
Rg. 31 is a diagram showing an embodiment of an 
ID panel in which an interpolation change in is 
selected frame according to the present invention is 
made; 

Rg. 32 is a diagram showing an embodiment of an 
ID panel in which an interpolation change in 
selected frame according to the present invention is 20 
made; 

Rg. 33 shows a flow chart in the case where infor- 
mation of the result of image detection is stored into 
the data structures of M-icons in the lowermost 
layer of a hierarchical structure; 25 
Fig. 34 shows a flow chart of a procedure for dis- 
tinctively displaying a zone having already been 
subjected to image detection processing and a 
zone having not yet been subjected to image detec- 
tion processing; 3^ 
Rg. 35 shows an example of a data structure con- 
cerning a hierarchical structure for attached M- 
icons; 

Rg. 36 shows an example of a data structure con- 
cerning a hierarchical structure for scenes; 35 
Rgs. 37A, 37B and 37C show an example of a data 
structure concerning a hierarchical structure for 
cuts: 

Rg. 38 shows another example of the data struc- 
ture concerning the hierarchical structure for cuts; 40 
Rg. 39 shows a further example of the data struc- 
ture concerning the hierarchical structure for cuts; 
Rg. 40 shows a furthermore example of the data 
structure concerning the hierarchical structure for 
cuts; and ^5 
Rg. 41 shows a still further example of the data 
structure concerning the hierarchical structure for 
cuts. 
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[0042] Embodiments of a metfiod and apparatus of 
the present invention will now be described in reference 
to the drawings. 

[0043] Rg. 3 is a block diagram showing an exam- 55 
pie of the construction of a motion picture editing appa- 
ratus according to the present invention. Reference 
numeral 3010 denotes a CPU (Central Processing 



Unit), numeral 3020 a memory, numeral 3030 a cut 
change point detecting unit, numeral 3040 an image 
reproducing device, numeral 3050 a video interface, 
numeral 3060 a random access information storage 
devic such as a magnetic disk, numeral 3070 a frame 
buffer, numeral 3080 a monitor, numeral 3090 an input 
device, numeral 3100 a bus, and numeral 3110 an 
image signal output terminal. 

[0044] This motion picture editing apparatus is pro- 
vided with the CPU 3010 for performing the control of 
each conrYx>nent of the motion picture editing apparatus 
and the edition under the control of software (or pro- 
gram), the monitor 3080 for displaying control informa- 
tion of the CPU 3010 and information concerning 
edition which includes information of a motion picture, 
scene images and cut images and information of a hier- 
archical structure (or free structure) representing the 
state of edition, the input device 3090 such as a mouse 
or keyboard for selecting information displayed on a 
screen of the monitor 3080 to make the input of a com- 
mand to tiie CPU 3010. the frame buffer memory 3070 
for storing image information to t>e displayed on the 
monitor 3080. the image refxodudng device 3040 such 
as a video tape recorder (VTR) for reading and repro- 
ducing from a recording medium an image signal of a 
motion picture to be used for edition and outputting it to 
the video interface 3050. the video interface 3050 for 
receiving the image signal of tiie motion picture inputted 
from the image reproducing device 3040 to convert it 
into image information having a format handled by the 
motion picture editing apparatus which information is 
then outputted to the information storage dev^ice 3060. 
the cut change point detecting unit 3030 and the mem- 
ory 3020 and for receiving image information of the 
motion picture edited with the format handled by the 
motion picture editing apparatus and outputted from the 
information storage device 3060 to convert it into an 
image signal which is then outputted from the image 
signal output terminal 3110. the cut change point 
detecting unit 3030 for comparing adjacent ones of 
frame images forming a motion picture to separate tinem 
from each other under the judgement of the adjacent 
frame images as belonging to different cut images in the 
case where a difference between the adjacent frame 
images has a change exceeding a predetermined quan- 
tity, the memory 3020 for storing software for control of 
the CPU 3010 and tenporarily storing various control 
information (for example, a series of frame numbers for 
a motion picture, frame numbers of detected cut change 
points. M-icons or images for the purpose of display to 
be displayed on the screen of the monitor 3080 and hav- 
ing a reduced image size. M-icon file names, cut image 
numbers serially applied to cut images, and sequence 
numbers representing the sequence of plural cut 
images forming scene images) and so forth used in tiie 
motion picture editing apparatus, th information stor- 
age device 3060 such as a hard disk device, the bus 
3100 used for tfie transfer of digital information between 
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the components in the motion picture editing apparatus, 
and th image signal output terminal 3 1 1 0 for outputting 
image information of the motion picture. The random 
access information storage device 3060 stores the 
image infbmiation of the motion picture reproduced by 
the image reproducing device 3040, converted by the 
video interface 3050 into a required format and used for 
edition, and stores edition information and control infor- 
mation (for example. M-icons) temporarily stored in the 
memory 3020 and used in the motion picture editing 
device. 

[0045] The bus 3100 is connected with the CPU 
3010. the memory 3020, the cut change point detecting 
unit 3030. the video interface 3050. the information stor- 
age device 3060, the frame buffer memory 3070, the 
monitor 3080 and the input device 3090 which con- 
structs the motion picture editing apparatus. The CPU 
3010 makes the transmission and reception of various 
signals for those devices which are in turn controlled by 
access signals from the CPU 3010. Also, an instruction 
to the CPU 3010 is inputted from the input device 3090 
such as a mouse or keyt>oard. 

[0046] The procedure of an operation for detecting 
a specified object or character from a motion picture 
source subjected to edition will be described using Figs. 
2. 4 and 13. 

[0047] Fig. 4 shows an editing window on the 
screen of the monitor 3080 (Fig. 3) of the non-linear 
editing apparatus. 

[0048] In order to detect an object or a character 
(for example, "X"). an operator uses the mouse pointer 
810 to select an M-icon corresponding to a cut or scene 
to be deleted from the editing area 101 0 or the material 
area 1020 by clicking the mouse. At this time, the M- 
ioon is brought into a selected condition. To thus bring 
an M-icon into a selected condition is called attach (or 
attaching the M-icon). A method of displaying the 
selected condition may include, for example, enclosing 
the selected M-icon by a thick frame, changing the dis- 
play color of the M-icon. and flickering the M-icon. 
[0049] For example. M-icons 1031 to 1033 in the 
material area 1020 shown in Fig. 4 are attached and the 
character detecting Ixjtton 1051 in the edition function 
button group 1050 is then clicked. As a result, a window 
shown in Fig. 2 is displayed on the monitor 3080. A 
processing for detecting the character "X" is performed 
on this window. Fig. 2 is a diagram for explaining a 
method in which the detection of a specified character is 
made with the features of that character defined. Refer- 
ence numeral 300 denotes an identify (ID) window, 
numeral 301 an OK button, numeral 302 a CANCEL 
button, numeral 303 a slider knob, numeral 304 a sliding 
cursor, numeral 305 an enlarged image, numeral 306 an 
enlargement display area, numeral 307 an OBJECT 
button, numeral 308 a color designating frame, numeral 
309 a SEARCH button, numeral 31 1 a still image view 
slider, numerals 321 to 330 frame images, numeral 809 
a film image, numeral 810 a mouse pointer, numeral 



815 a BEGIN button, and numeral 816 an END button. 
[0050] In Fig. 2, frame images 321 to 330 included 
in the attached M-icons 1031 to 1033 are displayed in 
the fym image 809. The frame innages 321 to 330 are 

5 arranged in order along a time axis. Right and left por- 
tions being not displayed in the window can be viewed 
by performing a processing for scrolling the slider knob 
303 of the still image view slider 311 right and left to 
slide displayed frame images. This saoll processing is 

10 enabled, for example, by moving the mouse pointer 810 
to the slider knob 303 and dragging the pointer by the 
mouse or attaching an arrow portion of the sliding cur- 
sor 304. An enlarged frame image 305 is displayed in 
the enlargement display area 306 at a left/lower portton 

15 of the ID window 300. The enlarged frame image 305 is 
an enlarged version of the leftmost one 321 of the frame 
images 321 to 330 in the film image 809. In order to per- 
form a processing for detecting the character "X", fea- 
tures to be detected as the character "X" are defined 

20 using the enlarged image 305. The frame image 321 
displayed at the leftmost and the enlarged image 305 
displayed in the enlargement display area 306 are inter- 
locked with each other. Therefore, the operator can 
scroll a frame image in the fHm image 809 so that it is 

25 displayed in the enlargement display area 306 as the 
enlarged image 305 which is to be used for the charac- 
ter "X" detection processing. While viewing the enlarged 
image 305 with the frame images in the film image 809 
scrolled, the operator selects a frame image which is 

30 regarded as being optimum in regard to the character 
"X" detection processing. Next, the combination of spe- 
cific colors possessed by the character "X" is desig- 
nated on the basis of the enlarged image 305. The 
frame image 321 of the frame images 321 to 330 is 

35 shown so that it is displayed with a simple image. The 
representation of images to be displayed as the other 
frame images 322 to 330 is omitted from the illustration 
of Fig, 2. 

[0051] A method of designating the combination of 
40 specific colors includes clicking the OBJECT tajtton 307 
and then designating a color designating frame 308 in 
the enlarged image 305 for defining an extent In which a 
specific color is designated. 

[0052] The designation of the color designating 
45 frame 308 is similar to a method of g^erating a square 
frame in the case where the drawing is made in a per- 
sonal computer or the like. It is of course that the move- 
ment and size change of the color designating frame 
308 after the designation are possible. 
so [0053] The color of a portion enclosed by the color 
designating frame 308 assumes a specific color pos- 
sessed by the character "X" (or a pixel region occupied 
by the character "X*' in the frame image). In the exanple 
shown in Fig. 2, the color of a dress worn by the charac- 
55 ter "X" is designated. The others, for example, the color 
of hair, the color of skin and so forth are designated. 
Data of the combination of these designated colors is 
used for the detection processing. 
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[0054] The character detecting function as men- 
tioned above can be provided using a method disclosed 
by. tor example. JP-A-6-223179. 
[0055] After the color designation is conpleted. the 
SEARCH button 309 is clicked to perform the detection 
processing for the M-icons 1031 to 1033. After the 
detection processing has been performed, the result of 
detection for the character "X" is displayed on the ID 
window 300, as shown in Fig. 13. The description of Rg. 
13 will be made later on. When the result of detection is 
to be held, the OK txitton 301 is clicked, thereby ctosing 
the ID window. When the result of detection is not to be 
held, the CANCEL button 302 is dtof^, thereby ck>sing 
the ID window. 

[0056] Two results are obtained t)y the character 
detecting function. One is information of a detection 
effectuating domain indicative of a coverage or domain 
throughout which the character "X" exists in an image 
material or cut. The other is position Information indicat- 
ing where is the character "X" located in frame images 
of the detection effectuating domain (for exanrple. left- 
side. central/rightskJe. upside. left/downskJe or the like). 
[0057] Under present conditions, however, the 
character detecting function may bring about erroneous 
detection or overlook owing to the precision of recogni- 
tion. Accordingly, it is preferable that tfie operator makes 
a manual confirmation or conectlon by use of the 
mouse while viewing the ID window. 
[0058] Fig. 8 shows an example of the editing win- 
dow on which the result of the atx}ve-mentioned charac- 
ter detection made with the M-icons 1031. 1032 and 
1033 selected in Fig. 4 is displayed. In Fig. 8, the same 
parts as those in Fig. 4 are denoted by the same refer- 
ence numerals as those used in Rg. 4. Numerals 12. 13 
and 14 denote index (IDX) icons. 
[0059] When ttie OK button 301 on the ID window 
300 shown in Fig. 2 is attached after the character 
detection is made, the ID window 300 is closed with the 
result of detection held and an IDX icon 12 is displayed 
at a lower portion of the material area 1020 of the edit- 
ing window 1000. Each time anotiier M-icon Is selected 
to make the character detection. IDX icons 13, 14. — 
are successively displayed. 

[0060] Next, an example of ihe GUI operation con- 
cerning the confirmation or correction of the result of 
character detection will be described by virtue of the 
window example shown in Fig. 13. Fig. 13 shows an 
exanple of display on the ID window in the case where 
the confirmation or correction of the result of character 
detection is made. In Fig. 13. the same parts as those in 
Fig. 2 are denoted by the same reference numerals as 
those used in Fig. 2. Numeral 80 denotes an ID window, 
numerals 800. 801 . 802. 802* and 803 detection arrows! 
numerals 804. 805. 806. 807 and 808 object frames, 
numeral 809' a film image, numerals 811 and 812 
detection effectuating domains, numerals 813, 814, 823 
and 824 frame images arranged In the film image 809* in 
order along the time axis from the left of tfie window. 



and numerals 817. 818. 820. 821 and 822 pixel regions 
which the character "X" occupies in the respective 
frame images. 

[0061] On the ID window 80 shown in Rg. 13, 

5 detection anows 800 to 803 displayed on the upper sKle 
of a film image 809' are arrows indicating tfie beginning 
and end of each detection effectuating domain. The 
detection arrow 800 or the detection arrow 802 is a 
detected head called a begin point. The detection arrow 

10 801 or the detection arrow 803 is a detected tennination 
called an end point. A domain sarxjwlched between the 
detection arrow 800 of the begin point and the detection 
arrow 801 of the end point is a detection effectuating 
domain 812. and a domain sandwiched between the 

15 detection arrow 802 of the begin point and the detection 
arrow 803 of the end point is a detection effectuating 
domain 81 1 . Each of object frames 804 to 808 displayed 
in frame images belonging to the detection effectuating 
domain 81 1 and the detection effectuating domain 812 

BO indicates a position at which the character "X" exists in 
tfie frame image. The object frame is displayed as the 
result of character detection. 

[0062] The change or correction for the detection 
effectuating domain and the object frame is made as 

25 mentioned below. First, the change of the detection 
effectuating domain will be described. 
[0063] Referring to Rg. 13. the description will be 
made in conjunction with the case where the detection 
arrow 802 Indicative of the begin point is moved into the 

30 frame 81 3 located on the left side by one frame to elon- 
gate the detection effectuating domain 81 1 . The mouse 
pointer 810 is GUI parts capable of being moved on the 
window by virtue of a mouse (not shown). The mouse 
pointer 810 may also be operated by a general pointing 

35 device other than the mouse. 

[0064] First, tfie mouse pointer 810 is moved to a 
BEGIN button 815 and the mouse is then clicked. Next, 
the nx)use pointer 810 is moved to tfie frame 813 and 
the mouse Is then clicked. Thereupon, a detection arrow 

40 802' representing a new begin point is displayed. There- 
after, the detection an-ow 802 representing tiie original 
begin point is clicked to extinguish the detection arrow 
802 of the original begin point. As a result, the detection 
effectuating domain 811 is enlarged leftwards by one 

45 frame. Similarly, a change in end point may be made by 
clicking an END button 816. clicking a frame for which 
the setting of a new end point is desired, and thereafter 
clicking a detection arrow of the original end point 
[0065] Next, the change or correction for tfie object 

50 frame will be described. This change too is made using 
the mouse pointer 810. 

[0066] The description referring to Rg. 13 will be 
made in conjunction with an example taken for the case 
where ttie object frame 806 of the frame image 814 is 
55 changed. In Rg. 13. the object frame 805 of the frame 
image 824 approximately coincides in size and position 
with a pixel region 818 representative of the character 
"X" whereas the ol^ject frame 806 of the frame image 
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814 has a size somewhat larger than a pixel region 820 
of the character "X" and a position nearer to the center 
than the pixel region 820. In this case, when seeing this 
frame image 81 4. the operator may t>e perplexed t>y the 
position of the object frame 806 with the result of erro- 
neous recognition that the pixel region of the character 
-X" resides at a central portion of the frame. Therefore, 
it is necessary to change or correct the size and position 
of the ol3ject frame 806 in order to make conformabiiity 
to the pixel region 820 of the character "X". The change 
in size or position of the object frame (804. 805. 806. 
807. 808) can be made by use of a method of deleting 
the present otsject frame and thereinstead setting a new 
frame as well as known methods, for example, screen 
operation and object operation in Windows 95 which is 
an operating system (OS) of Microsoft. 
[0067] in the case where there is performed the 
operation of changing the object frame 806 in conform- 
ity to the pixel region 820 of the character "X". all object 
frames succeeding the object frame 806 are similarly 
changed so that they have the same size and position 
as the changed object frame 806 or they have the same 
size and position as the pixel regions of the character 
"X" (for example, the pixel regions 821 and 822 of the 
character "X" in the object frames 807 and 808). 
[0068] When the CANCEL button 302 shown in Fig. 
2 is attached in the course of character detection or 
after the completion thereof, the ID window 300 is 
closed with the result of detection cancelled- On the 
other hand, when the OK Ixjtton 301 on the window 300 
is attached, the window 300 is cbsed with the result of 
detection held and an IDX icon 12 is displayed at the 
lower portion of the material area 1020 of the editing 
window 1000, as shown in Fig. 8. Information concern- 
ing the result of detection processing for the character 
"X" is stored in the data structure of the IDX icon 12. 
[0069] Such an IDX icon is generated one for all M- 
icons simultaneously attached in the editing window 
1000 for character detection. Namely, if different M- 
ioons are attached respective times for character detec- 
tion processing, new IDX icons 13. 14. — are succes- 
sively generated. 

[0070] When the IDX icon 12 is double-clicked by 
the mouse or the IDX icon 12 is selected and an IDX 
information button. 1053 is attached, an IDX window 
indicative of IDX information as shown in Fig. 6 is pop- 
up displayed. 

[0071] Fig. 6 shows an example of display on the 
IDX window in which the respective results of detection 
for characters "A", "B" and "C* are displayed. This corre- 
sponds to the result when the same group of M-icons 
1031 to 1033 is subjected to respective detection 
processings for three different characters. Reference 
numeral 600 denotes an IDX window, numeral 603 a 
detection zone display area, numerals 604. 605 and 606 
zones, numeral 61 1 a target display column, numeral 
614 a detection effectuating domain, numeral 615 a 
detection ineffectuating domain, numerals 631 , 632 and 



633 target images, numerals 634. 635 and 636 charac- 
ter's names, numerals 637, 638 and 639 detection 
domain bars, numeral 650 a detection result display col- 
umn, numeral 640 a slider for sliding the display con- 

5 tents of the detection result display column 650 in a 
lateral direction, numeral 641 a slider tor sliding the dis- 
play contents of the detection result display column 650 
in a longitudinal direction, numeral 810 a mouse pointer, 
and numeral 241 a CLOSE button. 

10 [0072] In Fig. 6. a detection zone display area 603 
on the IDX window 600 is displayed along a time axis 
with the lateral direction taken as a progressing direc- 
tion. Also, the detection zone display area 603 are ncMr 
divided into three zones 604 to 606. The division to 

15 these zones 604 to 606 is made in accordance with the 
cut length (or the number of frames) of each of the M- 
icons 1031 to 1033 attached by the operator. Namely, 
the number of frames of the M-icon 1031 corresponds 
to the number of frames of the zone 604, the number of 

20 frames of the M-icon 1032 corresponds to that of the 
zone 605. and the number of frames of the M-icon 1 033 
corresponds to that of the zone 606. Further, in a target 
display column 61 1 . the target images 63 1 . 632 and 633 
of detected characters and the character's names 634, 

25 645 and 636 thereof are displayed in the longitudinal 
direction. 

[0073] For example, in the case where a character 
"A" is detected, a target image 631 and a character's 
name 634 are displayed in a target display colunui 611 

30 and a detection domain bar 637 is displayed in a detec- 
tion result display column 650 on the right side of the 
target display column 61 1 . Further, in the case where a 
character "B" is detected, a target image 632 and a 
character's name 635 are displayed on the lower side of 

35 the character "A" in the target display column 61 1 and a 
detection domain bar 638 is displayed in the detection 
result display column 650 on the right side of the target 
display column 61 1 . Similarly, in the case where a char- 
acter "C is detected, a target image 633 and a charac- 

40 ter's name 636 are displayed on the lower side of the 
character "B" in the target display column 611 and a 
detection domain bar 639 is displayed in the detection 
result display column 650 on the right side of the target 
display column 61 1 . The target image 631 , 632 or 633 

45 is a frame image (for example, the frame image 305 in 
Fig. 2) in an enlarged display form used for acquiring 
specific colors to detect the character -A". "B" or "C. 
The character's names 634 to 636 under the target 
images 631 to 633 have the respective names of 

so ot)jects of detection displayed therein. Since the display 
column for the detection domain bars 637. 638 and 639 
is limited, there may be the case where a detection area 
is so large that the whole of the detection area cannot 
be displayed at once. In that case, a slider 640 is oper- 

55 ated to scroll the display in the lateral direction. Also, in 
the case where the number of detected characters is 
large, it is not possit>te to display all the detected char- 
acters at once. (In the example shown in Fig. 6. the 
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number of characters capable of being displayed at 
once is 3.) In this case, a slider 641 is operated to scroll 
the display in the longitudinal direction. 
[0074] As mentioned in the above, the area of the 
detection domain bars 637 to 639 includes the results of s 
detection of the characters "A", "B* and "C" carried out 
for the cuts or M-icons 1031 to 1033. 
{0075] Next, the result of detection will be described 
with the detection domain l>ar 637 taken as one exam- 
ple, 10 
[0076] A detection effectuating domain 614 indi- 
cates a domain where the character "A" is detected {or 
the detection of the character "A" is effectuated) in the 
zones 604. 605 and 606 with regard to ttie M-icons 
1031 to 1033. A detection ineffectuating domain 616 75 
indicates a domain where the character "A" is not 
detected (or the detection of tiie character ^A" is ineffec- 
tuated). For example, the zone 604 includes a time- 
sequential array of frame images collected in associa- 
tion with tile M-icon 1031 . the zone 605 includes a time- so 
sequerYtial array of frame images collected in associa- 
tion with the M-icon 1032, and the zone 606 includes a 
time-sequential array of frame images collected in asso- 
ciation with the M-icon 1033. The zone is represented 
with a predetermined width applied for each frame 2S 
image. In Fig. 6. the detection effectuating domain 614 
and the detection ineffectuating domain 616 are repre- 
sented in a distirx:tive manner so tiiat the former has an 
oblique line pattern and the latter has a white pattern. 
The representation of effectuation/ineffectuation of 3o 
detection may be aibitrary so long as the distinction 
from each other is provided. For example, tfie detection 
effectuating and ineffectuating domains 614 and 616 
may be displayed with orange and grey colors, respec- 
tively, or one of the domains 61 4 and 61 6 may be flick- 35 
ered. 

[0077] The detection effectuating domain 614 and 
the detection ineffectuating domain 616 are also dis- 
played on the side faces of tiie M-icons 1031 to 1033 
shown in Fig. 8. 40 
[0078] Rgs. 12A. 12B and 12C are views showing 
that the results of character detection are displayed on 
the side faces of tiie M-icons 1031 to 1033 shown in Fig. 
8. As in Rg. 6, zones 604, 605 and 606 are ones for the 
M-icons 1031. 1032 and 1033, respectively The width 45 
of the side face of the M-icon indicates the number of 
frames of that M-icon itself (or is represented with a 
fixed thickness lor each one frame image). For the 
detection effectuating domain 614. the side face of the 
M-icon 1031, 1032 or 1033 is displayed witti oblique so 
lines by the corresponding number of frames. For the 
detection ineffectuating domain 61 6. the side face of the 
M-icon is displayed witfi a white color by the corre- 
sponding number of frames. The display of such 
domains may be set such tiiat it is always made as the ss 
result of detection or is usually made in the case where 
the operator attaches the target image 631. Also, only 
one of the domains 614 and 616 may be displayed. 



[0079] Actually, each of the M-icons 1031 to 1033 
shO¥wi in Figs. 12A. 12B and 12C is displayed witii a 
reduced image for the purpose of display associated 
with that M-icon. However, such display is omitted from 
tfie illustration of Figs. 12A to 12C- 
[0080] When one of the detection effectuating 
domains 614 in. for exanrple. the zone 605 is attached, 
tiie pattern or color of the display of the attached detec- 
tion effectuating domain belonging to the zone 605 of 
the detection domain t>ar 637 is changed to, for exam- 
ple, a purple color and a purple spot light is directed 
therefrom towards the M-icon 1032 (Fig. 8) correspond- 
ing to the zone 605. This function makes it easy to find 
out which one of M-icons does a given detection effec- 
tuating domain belong to. 

[0081] In tiie case where the IDX window 600 
shown in Hg. 6 is to be closed, ttie CLOSE button 241 
is clicked. 

[0082] The character detecting/displaying metiiod 
described above has the following problems. 

(1) With the above-mentioned character detecting 
function, one IDX icon for attached M-icons is gen- 
erated in the material area. Namely, if different M- 
icons are attached respective times for character 
detection processing, tfiere results in that new IDX 
icons are successively generated. This makes it 
hard to see the material area. 

(2) An IDX icon has no information of which M-icon 
fias the detection processing for who of characters 
t>een performed for. Therefore, an operator is 
required to confirm such information by opening an 
IDX window corresponding to tiie IDX icon. Thus, a 
some time/laix)r is taken for confirmation. 

(3) There is not provided a function of deleting 
detection result information displayed on an IDX 
window for each character. Therefore, it is not pos- 
sible to delete an unnecessary detection result from 
the window. 

(4) Data of the combination of colors used the 
respective detection processings for characters "A". 
"B" and "C" is not preserved in the data structure of 
the corresponding IDX icon. Therefore, the comfc)i- 
nation of colors having already been used for a 
detection processing for a certain M-icon cannot be 
utilized in a detection processing for another M- 
icon. 

[0083] In embodiments of the present invention as 
will now t>e described, the above-mentioned problems 
are overcome, thereby providing an image editing/dis- 
playing method in which tfie implementation of a simple 
screen operation is possible and a high functionality is 
availat>le. 

[0084] In the image editing/displaying metiiod 
according to the emlxxiiments of the present inventk)n. 
detection information concerning a specified character 
stored in the data structure of an IDX icon in ttie atxyve- 
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menttoned method is stored in not the data structure of 
the IDX but the data structure an M-icon which will be 
mentioned later on. Thereby, it is not necessary to gen- 
erate IDX icons. 

[OOaS] An IDX window is displayed on the basis of 
detection information stored in the data structure of an 
M-icon. However, since attached M-icons may include 
ones having not yet been subjected to a detection 
processing for a specified image, there is provided a 
representation distinctively indicating whether or not the 
M-icon is one having been subjected to detection 
processing. Further, though the side face of an M-lcon 
in the above-mentioned method is provided with only 
the representation of a detection effectuating domain, 
the M-icon side face in the emtxxJiment of the present 
invention is provided with this representation as well as 
the representation indicating whether or not the M-icon 
is one having been subjected to detection processing. 
[0086] Also, a function of deleting the result of 
detection displayed for each specified object image is 
newly added. 

[0087] Further, there is provided a combining furK- 
tion with which after a detection processing has been 
performed for various objects, the results of detection 
for different color information (for example, the cases of 
different dress colors) concerning the same object are 
combined into one. As a function contrary to that, there 
is also provided an uncomt)ining function with which the 
combinative result of detection is restored to the original 
results of detection. 

[0088] There are further provided a preview func- 
tion with which only a detection effectuating domain 
obtained by a detection processing is reproduced in a 
motion picture form and an information function with 
which, for example, intonation of images subjected to 
detection processing is added to the result of detection 
and this is stored into the data structure. 
[0089] Also, there is provided a function with which 
the width of the time series of a detection zone display 
area can freely be set. Thereby, it is possitHe to adjust 
the width of display of a detection effectuating domain or 
the like freely. 

[0090] Further, data of the combination of colors 
used in the respective detection processings for the 
images of objects pf detection is preserved in the data 
structure of the corresponding M-icon so that it can be 
reused. 

[0091] An emtxxJiment of the present invention will 
now be described. The successive description of the 
procedure of operation referring to Figs. 4 to 10 will be 
made starting from a state in which a processing for 
detection of a specified character has never been per- 
formed. 

[0092] Start from the screen of Fig. 4 which shows 
an example of an editing window displayed on the 
screen of the monitor 3080 in the motion picture editing 
apparatus shown in Fig. 3. 

[0093] In order to detect a character "A**, an opera- 



tor first attaches M-icons 1031 to 1033 corresponding to 
cuts or scenes made an object of detection. Then, the 
operator attaches the character detecting button 1051 
in the edition function txjtton group 1050. As a result, an 
5 IDX window shown in Fig. 5 is displayed on the monitor 
3080. 

[0094] Fig. 5 shows an example of the IDX window 
in a state in an initial state or a state in which a character 
detection processing has not yet been performed. Ref- 

10 erence numeral 6000 denotes an IDX window, numerals 
237. 238 and 239 detection result display columns, 
numeral 240 a NEW button, numeral 241 a CLOSE but- 
ton, numeral 242 a SEARCH button, numeral 243 a 
COMBINE button, numeral 244 an UNGOMBINE but- 

15 ton. numeral 245 a PREVIEW button, numeral 246 an 
INFORMATION Ixjtton. numeral 247 a visual range 
adjusting slider, and numeral an ADJUST kxrtton. Other 
numerals are the same those used in Fig. 6. 
[0095] This stage is a state in which the detection 

20 processing has never been performed for the M-icons 
1031 to 1033. Therefore, neither the target display col- 
umn 61 1 nor the detection result display columns 237. 
238 and 239 on the IDX window 6000 has data dis- 
played therein, as shown \n Fig. 5. This state is an initial 

25 state. 

[0096] First, the detection processing is initiated for 
the character "A". 

[0097] When the operator dicks the NEW button 
240. an ID window 300 shown in Rg. 2 is displayed on 
30 the monitor 3080. On this ID window 300. the operator 
performs the detection processing for the character "A". 
The details of the operation for this detection processing 
have already been descrit)ed. 

[0098] After the detection processing is performed. 

35 the result of detection for the character "A" is displayed 
on the ID window 300. as shown in Fig. 1 3. If necessary, 
the operator makes a change or correction for the result 
of detection. Thereafter, the OK button 301 is clicked, 
thereby closing the ID window 300. 

40 [0099] At a point of time when the OK button 301 is 
clicked, data of the result of detection for the character 
"A" is stored into data structures corresponding to the 
attached M-icons 1031 to 1033. However, in the case 
where an upper layer of the tree structure, for example. 

45 an M-icon 1220 "a" (Fig. 8) is attached, the data of the 
result of detection is stored into data structures corre- 
sponding to M-icons 1210 "c" and "d" in the lower- 
most layer of the tree structure. The reason why the 
detection result data is always stored in the lowermost 

so M-icons is that the detection result data is prevented 
from being affected by an editing operation in the editing 
area 1010. Namely, if data of the result of detection for 
the lowermost layer M-icons 1210 "b". "c" and "d" is 
stored in the M-icon 1220 "a" of the upper layer of the 

55 tree structure, a change in detection result data may be 
required, for example, in the case where the M-icon 
1210 "c" is removed from the tree structure. The proce- 
dure for thus storing data of the result of detection 
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always in the information region of an M-icon of the low- 
ermost layer of the hierarchical structure will be 
described later on in reference to a flow chart shown in 
Fig. 33. 

[0100] When the ID window 300 is closed, there s 
returns to the display of the IDX window again. In this 
case, the result of detection concerning the character 
"A" is displayed on the IDX window, as shown in Fig. 7. 
Fig. 7 shows the IDX window on which only the result of 
detection concerning the character "A" is displayed. io 
Reference numeral 6001 denotes an IDX window, 
numeral 631 a target image of the character "A", 
numeral 634 a characters name, and numeral 637 a 
detection domain bar The other numerals are the same 
as those used in Fig. 5. 75 
[01011 When the CLOSE button 241 is clicked, 
there returns to the editing window 1000 shown in Rg. 
4. 

[0102] Further, the detection processing is also per- 
formed for characters "B" and "C". However, an M-icon 20 
made an object of detection is changed. By way of 
example, it is assumed that the M-icons 1031 and 1033 
are attached when the detection processing is to be per- 
formed for the character "B* and only the M-icon 1032 is 
attached when the detection processing is to be per- 25 
formed for the character "C". The description of an oper- 
ation of this detection processing will be omitted since 
the operation is similar to that mentioned above. 
[0103] After the detection processing for the char- 
acters "B" and "C* is completed, the M-icons 1031 to 30 
1033 are attached again to open the IDX window 600. 
An example of this state is shown in Fig. 1 . Fig. 1 shows 
an embodiment of the IDX window. In Fig. 1. parts hav- 
ing the same functions as those in Fig. 7 are denoted by 
the same reference numerals as those used in Rg. 7. 35 
Numeral 600 denotes an IDX window, and numerals 
250. 251 and 252 denote detection unsubjected zones 
which represent zones have not yet been subjected to 
character detection processing. 

[0104] The portions of the detection unsuljjected 40 
zones 250 to 252 represent frame information of M- 
icons in regard to which no detection processing has 
been performed. Namely, the detection unsubjected 
zone 251 indicates that no detection processing has 
been performed for the .character "C" in regard to the M- 45 
icon 1031 (or zone 604). This state will hereinafter 
refened to as "having not yet been sutsjeded to detec- 
tion processing*. To the contrary, a state in which the 
detection processing has already k>een performed will 
be referred to as ^'having already t>een subjected to so 
detection processing". Similarly, the detection unsub- 
jected zone 250 indicates that the M-icon 1032 (or zone 
605) has not yet been subjected to detection processing 
for the character "B". The detection unsubjected zone 
252 indicates that the M-icon 1033 (or zone 606) has ss 
not yet been subjected to detection processing for the 
character "C". Thereby, the distinctive representation of 
an M-icon having already been subjected to detection 



processing for a character made an otjject of detection 
and an M-icon having not yet been sul)jected to detec- 
tion processing therefor is enabled on the IDX window. 
[01 05] This representation of the state of "having 
not yet been subjected to detection processing" Is also 
reflected on the side face of an attached M-icon. For 
example, consider the result of detection for the charac- 
ter "B". An operation is performed by attaching the tar- 
get image 632. When the target image 632 is attached, 
the M-icon 1032 is displayed so that the whole of the 
side face thereof has a color representing the detection 
unsubjected zone 250 having not yet been subjected to 
detection processing, as shown in Fig. 9B, since the M- 
icon 1032 has not yet been sut3jected to detection 
processing for the character "8" and the width of the 
side face of the M-icon indicates the number of frames 
of the M-icon itself. Figs, 9A. 9B and 9C are views show- 
ing that the result of detection for the character "B" is 
displayed on the side faces of the M-icons 1031 to 1033 
shown in Rg. 4, As in the case of Rg. 6. zones 604. 605 
and 606 are ones for the M-icons 1031 . 1032 and 1033, 
respectively. 

[0106] Thereby, the representation of an M-icon 
having not yet been subjected to detection processing is 
also enabled on the editing window 1000. 
[0107] Further, since data of the combination of 
colors used in the detection processings for the charac- 
ter "A". "B" or "C" is presented in the data structure of an 
M-icon, it becomes possible to use this data for an M- 
icon portion having not yet been subjected to detection 
processing. By way of example, there will now be 
described the case where a detection processing is to 
be performed for the character "C" in regaid to zones 
having not yet been subjected to detection processing, 
that is. the M-icons 1031 and 1033, as indicated by the 
detection unsubjected zones 251 and 252. First, the tar- 
get image 633 is attached and a SEARCH button 242 is 
then clicked. As a result, the M-icons 1031 and 1033 are 
subjected to a detection processing by use of data of 
the combination of colors used in the processing for 
detection of the character "C". 

[0108] An example of the result of detection men- 
tioned in the above is shown in Fig. 10. Rg. 10 is a dia- 
gram showing an emtKxIiment of an IDX window 
according to the present invention. In Fig. 10. the same 
parts as those in Fig. 1 are denoted by the same refer- 
ence numerals as those used in Rg. 1 . Numeral 6003 
denotes an IDX wirKtow. numeral 280 a new detection 
effectuating domain representing a portion where the 
character "C" is detected, and numeral 282 a new 
detection ineffectuating domain representing a portion 
where the character "C" is not detected. Thus, the 
detection effectuating and ineffectuating domains 280 
and 282 are newly added. The procedure for distinc- 
tively displaying a zone having already been subjected 
to detection processing and a zone having not yet been 
subjected to detection processing will be described later 
on in reference to Figs. 34 and 35. 
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[01 09] This function provides the following nnerits. A 
first merit is that wh n a detection processing for the 
character "C*' is to be performed for the M-icons 1031 
and 1033, it is not necessary to designate the combina- 
tion of colors again. Also, since the preserved data of 
color combination is used as it is. the detection process- 
ing can be performed with the same corxiitton as that for 
the M-icon 1032. Further, since data of the result of 
detection is stored for each M-icon, it becomes possible 
to find out only the M-icons 1031 and 1033 having no 
result of detection concerning the character "C, thereby 
subjecting only such M-icons to detection processing. 
Thereby, it is possik)le to reduce a time required for the 
detection processing. 

[0110] Next, the desaiption will be made of the 
improvement in function of the IDX window. 
[0111] First, there is provided a comtnning function 
of combining the results of detection. This combining 
function is used, for example, in the case where pro- 
vided that the characters "A" and "B" are fellow friends, 
it is desired to handle the results of detection for the two 
characters in a form combined into one within the 
framework of fellow friends. There are two ways of com- 
binative handling, that is, a method in which a new 
detection effectuating domain is provided with frames 
where either the character "A" or the character "B" 
exists (OR condition) and a method in which a new 
detection effectuating domain is provided with frames 
where both the character "A* and the character "B" exist 
together (AND condition). An operation for this purpose 
is performed by clicking a COMBINE button 243 after 
the attachment of the target image 631 of the character 
"A" and the target image 632 of the character "B". Fig. 
1 1 is a diagram showing one embodiment of the combi- 
nation. In Fig. 1 1 . the same parts as those in Fig. 10 are 
denoted by the same reference numerals as those used 
in Fig. 10. Numeral 6002 denotes an IDX window, 
numeral 380 a detection effectuating domain, numeral 
383 a detection ineffectuating domain, numeral 387 a 
combine mark, numeral 661 a detection domain bar 
representing the result of detection of frames where 
either the cfiaracter "A" or the character "B" exists, and 
numeral 663 a target image. 

[01 1 2] In Fig. 1 1 . a detection domain bar 661 is dis- 
played with detection effectuating domains 380 in the 
case where frames containing either the character **A" 
or the character "B" are taken as a new detection effec- 
tuating domain. Also, a target image 663 is displayed 
with one 631 of the attached target images concerning 
the character "A" located at an upper position on the 
IDX window (Fig. 10). A combine mark 387 (for exam- 
ple. "C*^ representative of the combination is added to a 
left/upper portion of the target image. The result of 
detection of the character is displayed in a manner 
sited close upwards by one stage. 
[01 1 3] There is also provided an uncombining func- 
tion contrary to the combining function. The uncombin- 
ing function is a function of restoring the combinative 



detection result to the original results of detection. In 
operation for this purpose, the combined target image 
633 is attached and a UNCOMBINE button 244 is there- 
after clicked by the mouse. As a result, there returns to 

5 the state shown in Fig. 1 . 

[0114] With the combining and uncombining func- 
tions, the results of detection individually provided for 
respective characters can be arranged or integrated in 
terms of abstract contents of. for example, lellow 

10 frierxjs". 

[01 1 5] Next, a detection result deleting function will 
be described. This function is a function of deleting an 
unnecessary result of detectk>n from the structure data 
of an M-kx>n. In an operation for this purpose, for exam- 

15 pie, in the case where the result of detection of the char- 
act^ "A" is to be deleted, the target image 631 (Fig. 1) 
is attached and a DELETE key of the keyt>oard 3090 is 
thereafter depressed. Thereby, the result of detection 
concerning the character **A** is deleted from tiie data 

20 Structure of the M-icon as well as the IDX window. And, 
the remaining results of detection conceming the char- 
acters "B" and "C" are displayed on the IDX window in a 
manner sited close upwards. 

[0116] Next, a preview function will be described. 
25 This preview function is a function of movie-reproducing 
only a detection effectuating zone obtained by a detec- 
tion processing. In an operation for this purpose, for 
example, in the case where detection effectuating 
zones for the character "A" are to be previewed, the tar- 
so get image 631 of the character "A" is attached and a 
PREVIEW button 245 is thereafter clicked. As a result, 
a motion picture reproducing window is displayed in the 
editing window 1000 and only tiie frames of detection 
effectuating domains 214 to 215 are previewed by the 
35 reproduction. Thereby, it is possitjie to confirm tiie 
detection effectuating domain by means of a motion pic- 
ture. 

[0117] Next, an information function will be 
descrit^ed. This information function is a function with 
40 which information such as information of a character 
subjected to detection processing is added to the result 
of detection and this information added detection result 
is stored into the data structure of an M-icon. in an oper- 
ation for this purpose, for example, in the case where it 
45 is desired to see or record information concerning the 
character "A", the target image 631 of the character "A" 
is attached and an INFORMATION button 246 is there- 
after clicked. Thereby it is possible to add information 
(for example, name, birth date and so forth) for the char- 
so acter "A" to the result of detection of the character "A". 
[0118] A detection result adjusting function is per- 
formed, for example, in the case where the result of 
detection of the character "A" is to be adjusted. This is 
performed by attaching tiie target image 631 of the 
55 character "A" and thereafter clicking an ADJUST button 
248. 

[01 19] Finally, there will be described a function of 
adjusting a visible range of the result of detection dis- 
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played in the detection domain bar shown in Rg. 1 . This 
acjustment of the visible range of the result of detection 
is made by dragging a visible range adjusting slider 247. 
In the cas where a numeral displayed on the visible 
range acyusting slider 247 is "1 the visible range on the 5 
detection domain bar is one second, that is. 30 frames. 
By moving the visible range adjusting slider 247. it is 
possible to freely changing the display width of a detec- 
tion effectuating domain or the number of frames of an 
M-icon. With the use of this function, however large the 10 
number of frames possessed by an M-tcon to be dis- 
played is. the M-icon can be displayed with a display 
width reduced to allow to take a look at it. Also, even a 
detection effectuating domain having only one frame 
can be displayed with a display width enlarged. The unit 15 
of a numeral displayed on the visible range adjusting 
slider 247 can be set art>itrarily. For exanple, it may be 
the number of frames. 

[0120] In the foregoing emljodiment, one example 
of the image detecting method has been descrfced in 20 
conjunction with an example in which a character is 
detected. However, it is self-evident that an object 
image other than characters can be detected by desig- 
nating the combination of specific colors. Further, it is 
needless to say that the present invention can also be 2S 
applied in conjunction with a detecting method other 
than that mentioned at>ove. 

[0121] According to the emtxxdiment of the present 
invention mentioned atxjve. the result of a detection 
processing is stored in the data structure of an M-icon. 30 
Thereby, the representation of an M-icon having already 
been subjected to detection processing and the repre- 
sentation of an M-icon having not yet been subjected to 
detection processing are enabled on an IDX window 
and on the side face of the M-icon. 35 
10122] A second effect of the embodiment of the 
present invention is that with the construction in which 
data of the combination of colors used in a detection 
processing is preserved in the data structure of an M- 
icon. it becomes possible to perform a detection 40 
processing for only another M-icon which has not yet 
been subjected to detection processing. 
[0123] A third effect of the enrtKxJiment of the 
present invention is that the improvement in functional- 
ity of an IDX window is. attained by newly adding a com- 45 
bining/uncombining function, a deleting function, a 
preview function, an information function and a visible 
range adjusting function for the result of detection. 
[0124] Rg. 14 shows another exanple of the char- 
acter detecting window. Reference numeral 500 so 
denotes an ID panel, numeral 501 a film image, numer- 
als 521 to 530 frame images arranged and displayed on 
the film image 501 time-sequentially from the left of the 
screen to the right thereof, numeral 520 an enlarged 
version of the leftmost frame image 521 in the film 55 
image, numeral 502 an enlargement display area for 
displaying the enlarged image 520, numerals 503. 504 
and 505 color designating frames, numeral 508 a still 



image view slider, numeral 509 a slider knob of the still 
image view slider, numeral 532 a sliding cursor, numeral 
510 a display frame unit changing slider, numeral 511 
an AUTOMATIC SEARCH button, numeral 512 a detec- 
tion effectuating domain ADD button, numeral 514 a 
detection effectuating domain CLEAR txjtton, numeral 
541 an OK button, numeral 542 a CANCEL button, 
numeral 545 an OBJECT button, numeral 547 an object 
name enter column, and numeral 810 a mouse pointer. 
[01 25] In Fig. 1 4, frame images 521 to 330 as a part 
of frame images included in attached M-ioons 1030. 
1034 and 103A (Fig. 4) are displayed in the film image 
501 . The frame images 521 to 530 are arranged in order 
along a time axis. Right and left portions kjeing not dis- 
played in the window can be viewed by performing a 
processing for scrolling the slider knob 509 of the still 
image view slkler 508 right and left to slide frame 
images. This right/left scroll processing is enabled, for 
example, by moving the mouse pointer 810 to the slider 
knob 509 and dragging the mouse pointer by the mouse 
or attaching an arrow portion of the slider cursor 532. 
The display frame unit changing slider 510 is a slider for 
setting every what frames are the frame images 521 to 
530 in the film image to be displayed. In an example 
shown in Rg. 14. since a value set by the display frame 
unit changing slider 510 is "1", the frame images are 
displayed every frame. The set value can be changed 
up to. for example. "90". If the value is set to "90". the 
frame images will be displayed every 90 frames so that 
10 frame images displayed in the film image 501 repre- 
sent a part of 900 frame images. 
[01 26] An enlarged frame image 520 is displayed in 
the enlargement display area 502 at a left/lower portion 
of the ID panel 500. This enlarged frame image 520 is 
an enlarged version of the leftmost one 521 of the frame 
images 521 to 530 in the film image 501 . In order to per- 
form a processing for detecting a character "X". features 
to be detected as the diaracter "X** are defined using 
the enlarged image 520. The frame image 521 dis- 
played at the leftmost and the enlarged image 520 are 
interlocked with each other. Therefore, an operator can 
scroll a frame image in the film image 501 so that it is 
displayed in the enlargement display area 502 as the 
enlarged image 520 which is to be used for the charac- 
ter "X" detection processing. While viewing the enlarged 
image 520 with the frame images in the film image 501 
scrolled, the operator selects a frame image which is 
regarded as being optimum in regard to the character 
"X" detection processing. Next, the combination of spe- 
cific colors possessed by the character "X" is desig- 
nated on the tjasis of the enlarged image 520. In Fig. 
14. the frame image 521 of the frame images 521 to 530 
is shown so that it is displayed with a simple image. The 
representation of images to be displayed as the other 
frame images 522 to 530 is omitted from the illustration 
of Fig. 14. 

[0127] A method of designating the combination of 
specific colors includes attaching the OBJECT button 
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545 and then designating that color designating frames 
503 to 505 in the enlarged image 520 each of which 
defines an extent for designating a specific color. The 
number of the color designating frames may be any 
value equal to or larger than one. The size and the torm 
of the color designating frame can be designated. The 
others of the window shown in Fig. 14 are similar to the 
window shown in Fig. 2. 

[0128] Next, another enftediment of the present 
invention will be described. In the foregoing embodi- 
ment, it is necessary to operating three windows (or the 
ID panel, the IDX panel and the editing window) for 
character detection. However, there is a problem that 
since these windows are individually displayed on the 
monitor in the case where a function established (or 
built in) for each window is set or performed, it is not 
possible to make occasional confirmation of a relation- 
ship between the panels and a relationship between the 
material area and the editing area in the ecBting window 
on which the edition of a motion picture itself is per- 
formed. 

[0129] The embodiment descrifc>ed in below solves 
the above problem, thereby providing an editing method 
in which the result of generation and correction of data 
on an ID panel arxj an editing widow can be recognized 
on an IDX panel at a look. 

[0130] In the motion picture editing method of the 
present embodiment, two panels and an editing window 
as mentioned above are always displayed. Further, a 
processing on one off the two panels is simultaneously 
r^lected on the other panel and also displayed on M- 
icons on the editing window, thereby enabling the reali- 
zation of a motion picture editing method which Is used 
in a motion picture editing apparatus for selecting icons 
as image materials displayed on a display screen to 
make edition based on a screen display operation arxi 
in which when M-icons as image materials to be edited 
are selected, the sequence of combination in the edition 
is displayed together with the M-icons. thereby facilitat- 
ing an editing work. 

[01 31 ] Now. the mutual operation of an ID panel, an 
IDX panel, an editing area 1010 and a material area 
1 020 in a character detecting function according to the 
present invention will be described in accordance with 
the actual procedure of a detection processing. 
[0132] When an operator selects M-icons 1030, 
1034 and 103A on the editing window 1000 of Fig. 4 
having already been described, this selection can be 
recognized in a form in which the selected M-icons 
1030. 1034 and 103 A are endosed by thick color 
frames (for example, red solid lines), as shown in Fig. 
15. 

[01 33] Fig. 1 5 is a diagram showing an embodiment 
of the editing window when M-icons are selected. In Fig. 
15. the same parts as those in Fig. 4 are denoted by the 
same reference numerals as those used in Fig. 4. 
Numeral 1000 denotes an editing window. Thus, the 
operator first selects M-ioons in a section which is to be 



SL^jected to a processing for detection of a character. In 
this case, the number of selected M-icons may be plu- 
ral. The selected M-icons may be ones in either the edit- 
ing area 1010 or the material ar a 1020 or over both 
5 these areas. They may be upper M-kwis in a tree struc- 
ture. 

[01 34] Next a character detecting button 1 051 in an 
edition function button switch group 1050 is clicked. 
Then, an ID panel is displayed on the monitor 3080 so 

10 that it overlaps a part or the whole of the editing window 
1000. Fig. 16 is a diagram showing an example in which 
an ID panel 500' is displayed to cover the editing win- 
dow 1000 fully. Fig. 16 shows an initial state of the ID 
panel. In Fig. 16. the same parts as those in the ID 

75 panel shown in Fig. 14 are denoted by the same refer- 
ence numerals as tixjse in Fig. 14. Numeral 500' 
denotes an ID panel. 

[0135] In Fig. 16, the frame images of ten leading 
frames are displayed as a film image 501 on the ID 

20 panel 500* which is the display window of the ID panel in 
the initial state. Also, a frame image designated by th 
operator is displayed in an enlargement display area 
502. In Fig. 16, the illustration of each frame image as 
the film image 501 is omitted. 

25 [0136] In a first step for detection, the operator 
selects a desired frame image (in which a character to 
be detected exists) from the film image 501 displayed 
on the ID panel 500* so that it is dragged and dropped to 
the enlargement display area 502. 

30 [01 37] At this time, a frame image corresponding to 
the selected frame and read from the information stor- 
age device 3060 (or a version thereof trimmed into a 
proper size) is displayed as an enlarged image 520 in 
the enlargement display area 502. as shown in Fig. 14. 

35 Next, the operator selects color designating frames 503. 
504 and 505 indicative of the extents for designation of 
specific cotors with respect to tiie character "X" as an 
ot^ect of detection from the enlarged image 520 and 
turns on an AUTOMATIC SEARCH button 511. thereby 

40 initiating a detection processing. Since the operation of 
the detection processing has already been described, 
the description thereof will be omitted. 
[01 38] Fig. 1 7 is a diagram showing an embodiment 
of the ID panel of the present invention. It is shown that 

45 object frames are additionally displayed on the ID panel 
shown in Fig. 14. In Fig. 17, the same parts as those in 
Rg. 14 are denoted by the same reference numerals as 
those used in Fig. 14. Numeral 500" denotes an ID 
panel, numeral 531 a detection effectuating domain, 

50 numeral 506 a detection arrow representing a begin 
point of the detection effectuating domain 531 . numeral 
507 a detection arrow representing an end point of the 
detection effectuating domain 531 , and numerals 561 to 
563 object frames. 

55 [01 39] Namely, when a detection processing is per- 
formed on the ID panel 500 shown in Fig. 14, the detec- 
tion processing is performed for each frame to be 
subjected to detection so that regions having a charac- 
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ter "X" therein and their frame numbers are stored. 
When the detection processing is conrpleted. frames 
resulting in the effectuation of detection of the character 
"X" are sandwiched between a detection arrow 506 (or 
a begin point) and a detection arrow 507 (an end point) 5 
on a film image 501 of the ID panel 500". as shown in 
Fig. 17. Further, pixel regions having the character "X" 
therein are displayed in a form enclosed by object 
frames 561 to 563. In Fig. 17. frame Images resulting in 
the ineffectuation of detection of the character "X" are 10 
omitted from the illustration. 

[0140] In Fig. 17, if the result of detection of the 
character "X" is satisfactory, the operator turns on an 
OK button 541 so that the ID panel 500" is displayed 
with a reduced size at a predetermined location on the 75 
monitor 3080. for example, a left/ijpper portion thereof. 
At the same time, an IDX panel 600 as shown in Fig. 6 
is displayed covering the editing window 1000 and the 
ID panel. On this IDX panel 600. there is displayed the 
result of the character detection processing performed so 
in the above. After the confirmation of the result of 
detection is completed in this state, a CLOSE button 
241 is turned on so that the ID panel is displayed with a 
reduced size at a predetermined location on the monitor 
3080. for example, a right/upper corner portion thereof. 2s 
[0141] Though each of the ID panel and the IDX 
panel is displayed on tiie monitor 3080 in a reduced 
state, its display size can arbitrarily changed (enlarged 
or reduced). For example, in order that for the conven- 
ience of an operation only a part of tiie panel can be 30 
seen, it may be displayed in an enlarged form. After the 
completion of the operation, tiie panel may be displayed 
on a predetermined location of the monitor 3080 with an 
reduced size mentioned akx>ve. 

[0142] An example of the multiple window display in 35 
this state is shown in Rg. 18. Fig. 18 is a diagram show- 
ing an embodiment of the present embodiment in which 
an ID panel, an IDX panel and an editing window (that 
is. an editing area and a material area) are simultane- 
ously displayed on the monitor 3080. Reference 40 
numeral 1000 denotes the editing window, numeral 
1010 the editing area, numeral 1020 the material area, 
numeral 500 the ID panel, and numeral 600 the IDX 
panel. 

[01431 Since the display contents of the editing win- 45 
dow. tiie ID panel and the IDX panel shown in Fig. 18 
are changed, on an editing operation performed by the 
operator, in accordance with the contents of tiie editing 
operation, tiie description of the details thereof will be 
omitted. so 
[0144] Rrst. the description concerning Rg. 18 will 
be made in reference to the ecfiting window 1 000 shown 
in Rg. 4. the ID panel 500* shown in Fig 16 and the IDX 
panel 6000 shown in Fig. 5. 

[0145] In Fig. 18. the editing window 1000 shown in ss 
Fig. 4 is displayed on the monitor 3080. Furtfier, the ID 
panel 500* shown in Fig. 16 and the IDX panel 6000 
shown in Fig. 5 are displayed as an ID panel 500 and an 



IDX panel 600 so that they overlap the editing window 
1 000 at an upper portion thereof. 
[01461 When the operator selects M-icons 1030. 
1034 and 103A on ttie editing window 1000 showvn in 
Rg. 4. tiie selected M-icons 1030. 1034 and 103 A are 
enclosed by thick color frames (for example, flickered 
red sdki lines), as shown in Rg. 15. Thus, the selection 
of tiiose M-icons can be recognized at a look. Simulta- 
neously the display of the ID panel changes to the ID 
panel 500 shown in Rg. 14. Also, the frame images of 
ten leading frames are displayed as a film image 501 . 
[01 47] Next, in the case where all the M-icons 1 030. 
1034 and 103 A selected by the operator have never 
been subjected to detection processing, the IDX panel 
6000 shown in Rg. 18 is such that target image display 
areas 291. 292 and 293 and detection result display 
areas 237. 238 and 239 are all blank, as shown by the 
IDX panel 6000 of Rg. 5. 

[0148] In the case where the detection is to be per- 
formed, a desired frame image 521 containing a charac- 
ter or object to be detected is selected from the fflm 
image 501 displayed on tiie ID panel 500. as has 
already been described in conjunction witii Figs. 14 and 
1 6. so that it is dragged and dropped to tiie enlargement 
display area 502. And. a frame image read from the 
information storage device 3060 (or a version thereof 
trimmed into a proper size) is displayed as an enlarged 
image 520 in the enlargement display area 502. Next, 
the operator selects color designating frames 503. 504 
and 505 indicative of the extents of designation of spe- 
cific colors for an image as an object of detection (for 
example, ttie character "X" in this case) from tiie 
enlarged image 520 and turns on an AUTOMATIC 
SEARCH button 511, thereby initiating a detection 
processing. In the case where the display window of the 
ID panel 500 is so small that it is hard to see. the ID 
panel 500 is displayed with a properly enlarged size. 
[0149] Here, the frame image 521 dragged and 
dropped to the enlargement display area 502 is the tar- 
get image 631. This frame image is displayed as tiie 
synoptical display area 61 1 for the target image 631 on 
the IDX panel 600 having already been described in 
conjunction with Rg. 6. 

[01 50] With tiie detection processing performed for 
respective frames to be subjected thereto, regions hav- 
ing tiie character "X* therein and their frame numbers 
are stored. As tfie result of the detection processing, 
frames resulting in the effectuating of detection of the 
character "X" are sandwiched between a detection 
an-ow 506 (or a begin point) and a detection arrow 507 
(an end point) on the film image 501 of the ID panel 
500". as shown in Fig. 17. Further, pixel regions having 
the character "X" therein are displayed in a form 
enclosed by object frames 561 to 563. 
[01 51 1 At tiie same time, the detected conditions or 
the result of detection mentioned above are reflected by 
the IDX panel 600 and the editing window 1000 (Fig. 18) 
in accordance with the present invention. 
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[0152] Namely, the editing window 1000 of Fig. 18 
assunries an editing window, as shown in Fig. 8, on 
which an IDX Icon 12 is generated and displayed. Also, 
the result of detection is displayed on the side faces of 
the M-icons 1030. 1034 and 103A. as shown in Rgs. 
12A. 12Band12C. 

[0153] Further, the IDX panel 600 of Fig. 18 
assumes an IDX |3anel as shown in Fig. 19. The IDX 
panel shown in Fig. 1 9 shows an embodiment of the dis- 
play on the IDX panel 600 shown in Rg. 18. In Fig. 19. 
the same parts as those in Figs. 5 and 6 are denoted by 
the reference numerals as those used in Figs. 5 and 6. 
Numeral 614' denotes the representation of a newly 
added detiection effectuating domain area, and numeral 
616' denotes the representation of a deleted detection 
effectuating domain area The area representations 
61 4* and 616* will be descrttsed later on. In Fig. 1 9. a tar- 
get image 631 is displayed in a target image display 
area 611 while the conditions of frames subjected to 
detection processing are displayed in a detection 
domain t^r 637 in units of one frame. Namely, in the 
detection domain bar 637 including divisional zones 
604. 605 and 606 in units of the selected M-icons 1030. 
1034 and 103A, a domain 614 resulting in the effectua- 
tion of detection of the character "X" is showily (or heav- 
ily) colored or pattemed in units of one frame, thereby 
representing frame positions at which the character "X" 
exists. 

(01 54] With the construction, as shown in Fig. 1 8, in 
which the ID panel 500 and the IDX panel 600 are dis- 
played together with the editing window 1000 to reflect 
the detected conditions or the result of detection into 
other information display columns, it is possible to make 
confirmation in M-icon units (on the IDX panel) and in 
frame units (on the ID panel and the IDX panel). 
[0155] Next, the description will be made of a dis- 
playing method in the case where the result of detection 
is to be corrected. Though a correcting work may 
include the correction of the detection effectuating 
domain 531 shown in Fig. 1 7 (that is, the domain sand- 
wiched between the detection an'ows 506 and 507) and 
the correction of the object frames 561. 562 and 563. 
the correction of the detection effectuating domain will 
be made in the case of linkage display This work is per- 
formed in such a manner that on the ID panel 500" of 
Fig. 17 displayed as the ID panel 500 in Fig. 18. frames 
to be subjected to correction are displayed in the film 
image 501 arxJ the detection arrow 506 indicative of a 
begin point or the detection arrow 507 indicative of an 
end point is then dragged by tiie mouse pointer 810. 
When this con-ection work is performed, tiie IDX 600 
displayed in Fig. 18 togetiier with the ID panel 500 
assumes the IDX panel 600 of Fig. 19 in which the 
detection effectuating domain 614 for the character "X" 
is being displayed in the detection domain bar 637. 
Therefore, if any frame position in the zones of M-icons 
subjected to the detection processing in the detection 
domain bar is double-dtcked by the nxsuse. it is possible 



to display image data of the corresponding frame so 
that it is located at ttie head of the film image 501 (or the . 
left extreme of the ID panel). As a result, a work of dis- 
playing a desired frame on the film image in order to 

5 correct or confirm the result of detection is facilitated. 
This also holds for ttie case where a detection effectuat- 
ing domain is newly added. At this time, a beam repre- 
sentation 10 as shown in Fig. 20 is applied for the 
corresponding M-tcon on tiie editing window 1000. 

10 [01 56] Next, the operation on the ID panel 500 side 
will be described in more detail. 
[01 57] First, there will be described tiie case where 
the detection effectuating domain 531 shown in Fig. 17 
is con-ected. This is realized by moving the mouse 

15 pointer 810 to move the detection arrow 506 indicative 
of a begin point or the detection arrow 507 indicative of 
an end point, as has already been mentioned. At tiiis 
time, the showily colored or patterned domain 614 indi- 
cating detection effectuating frames in the detection 

20 domain t>ar 637 (Fig. 19) on ttie IDX panel 600 changes 
in compliance witti the movement of the detection arrow 
506 or 507 caused by the operator. 
[0158] Next, the description will be made of the 
case where the operator adds a new detection effectu- 

25 ating domain due to an omission in detection or the like. 
[0159] To perform this work, the <^erator clicks (or 
drags) and selects desired frame images on the film 
image 501 as a domain to t>e added and tiien turns on 
a detection effectuating domain ADD button 512. In 

30 compliance with the clicking (or dragging) motion of the 
mouse pointer 810. tiie condition of the selected domain 
(or frame images) is reflected and displayed in the 
detection domain bar 637 on the IDX panel 600. This 
area representation 614* in the detection domain bar 

35 637 is provided with another color, another pattern, the 
combination thereof or the like in order that it is distinc- 
tive from the area representation 614 in the case where 
tiie detection effectuating frames are showily colored or 
patterned. Thereby, at a look and more simply without 

40 making the switching of window display, the operator 
can know about which one of frames for which one of M- 
icons is the selection/change now made for. It is hard to 
recognize this information from tiie film image 501 
alone. 

45 [01 60] The above similarly holds for the case where 
the result of detection is to be deleted or cleared. In this 
case, a detection effectuating domain CLEAR button 
514 is turned on in lieu of the detection effectuating 
domain ADD button 512. At tiiis time, the area repre- 
ss sentation 616' of a deleted detection effectuating 
domain in the detection domain bar 637 is provided witfi 
a different color, a different pattern, tiie combination 
thereof or the like in order that it is distinguishatrfe from 
the representation in the case where detection effectu- 
55 ating frames are showily colored or patterned and the 
representation in the case where a detection effectuat- 
ing domain is added. 

[01 61 ] Next, a f urtiier embodiment will be described 
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in conjunction with the case where the detection 
processing was previously perfomied one or more 
times. 

[0162] Herein, the description will be made in con- 
nection to the case where a tree structure 1 040 (Fig. 21) s 
having already been subjected to editing work is 
invoked from the information storage device 3060. Fig. 
21 is a diagram showing an editing window in an 
embodiment of the present Invention. In Rg. 21. the 
same parts as those in Rg. 8 are denoted by the same io 
reference numerals as those used in Rg. 8, Numeral 
1000' denotes an editing window, numeral 1 1 an IDX 
icon, and numerals 121A, 121B, 121C. — . 121H, 1211. 
— M-icons representing the lowermost cut images in an 
editing area 1010. Also, it is assumed that tiiis tree is 
structure 1040 was previously subjected to detection 
processing once with the uppermost M-icon 1240 
s lected (that is, with all the M-icons of the lowermost 
layer taken as a section to be subjected to detection 
processing). 20 
[0163] Rrst. the description will be made of the 
case where a character detecting button 1051 is turned 
on with the uppermost M-icon 1240 selected for the tree 
structure 1040. As shown in Fig. 20. an ID panel 500 
and an IDX panel 600 are displayed on an editing win- 2S 
dow 1000 of the monitor 3080. At this time, ten leading 
frames of the M-icon 121 A at the head (or left extreme) 
of the lowermost layer shown in Fig. 21 are displayed as 
a film image 501 on the ID panel 500. On the IDX panel 
600, a target image 631 designated at the time of detec- 30 
Hon is displayed in a target image synopsis display area 
61 1 and the conditions of frames subjected to detection 
processing are displayed in a detection result bar 637. 
as shown in Fig. 19. 

(01 64] In this state, however, tfie result of the previ- 35 
ous detection processing is not displayed on the ID 
panel 500. In order to display this detection result, a 
desired target image 631 (Fig. 19) on the IDX panel 600 
is clicked. Thereby, image data of the frame of the target 
image 631 and color designating frames 503. 504 and 40 
505 used for color designation are simultaneously dis- 
played in an enlargement display area 502 (Rg. 14) on 
the ID panel 500. Further, a detection arrow 506 indica- 
tive of a begin point and a detection an-ow 507 indicative 
of an end point are displayed in the film image 501. 45 
Frame images 521 . 522 and 523 between tfie detection 
arrows 506 and 507 are displayed with respective object 
frames 561. 562 and 563. Also, the clicking of the target 
image 631 (Fig. 19) results in that M-icons 121 A. 121 B. 
121c, 121E and 121G having effectuated the detection so 
of the character "X" are displayed, for exanple. in a form 
enclosed by a red frame of dotted line (Fig. 21). This is 
convenient for. e.g. a work of gatiiering only the M-icons 
121 A. 121 B. 121c. 121E and 121G having the charac- 
t r "X" therein to generate a digest version. In Fig. 21. 55 
M-icons 1030. 1034 and 103A are displayed with red 
frames, too. This corresponds to the case where these 
M-icons 1030, 1034 and 103 A are respectively copied 



to tiie M-icons 121 A. 1218 and 121C in the editing area 
1010, The otiier M-icons 121D. — . 121K 1211, — are 
onesr suiting from the movement 
[0165] Next, consider tiie case where an M-icon is 
newly added from ttie material area in a state in which 
tfie tree structure 1040 is invoked. Figs. 22 and 20 are 
diagrams showing an editing window in an embodiment 
of the present invention. In Rg. 22. the same parts as 
those in Fig. 21 are denoted by the same reference 
numerals as ttiose used in Fig. 21. Numeral 1001 
denotes an editing window, numeral 1040' a tree struc- 
ture in the editing area 1020. numeral 121 Z an M-icon 
newly inserted between the M-icon 121 A and 121 B. 
numeral ir an IDX icon, and numeral 1053 an IDX 
information button. Rg. 20 stxjws an embodiment in 
which a beam representation 10 is additionally dis- 
played on the display window shown in Fig. 18. Rg. 23 
shows an example of the case where a new zone is 
added after the M-icon 103A for the zone 604. Fig. 23 
shows an IDX panel in an embodiment of the present 
invention as an example of display of the IDX panel 
shown in Rg. 18. In Rg. 23. the same parts as tiiose in 
Fig, 1 9 are denoted by tiie same reference numerals as 
ttiose used in Fig. 19. Numeral 6001 denotes an IDX 
panel, numeral 607 a newly added zone, and numeral 
608 a detection unprocessed zone (or a zone having not 
yet been subjected to detection processing) in the 
detection domain bar 637 indicating the location of 
newly added frame images. Fig. 24 is a diagram show- 
ing an ID panel in an embodiment of the present inven- 
tion. In Fig. 24. tiie same parts as those in Fig. 16 are 
denoted by the same reference numerals as tiiose used 
in Rg. 16. Numeral 5000 denotes an ID panel, numeral 
901 a detection unprocessed zone, numeral 902 a film 
image, numeral 903 a detection anow. and numeral 904 
a frame portion. 

[0166] In Rg. 22. since the detection unprocessed 
M-icon 121Z is inserted between the M-icons 121 A and 
121 B in the course of edition of a motion picture, the 
tree structure 1040' has tiie coexistence of M-icons hav- 
ing already been subjected to detection processing for a 
certain character "X" (that is, the zones 604. 605 and 
606 having already been subjected to the detection of 
the character "X") and an M-icon having not yet been 
subjected to detection processing (tfiat is. tiie zone 607 
having not yet been subjected to the detection of tiie 
character "X"). When tiie character detecting button 
1051 is turned on in this state, tiie zone for tiie M-icon 
121Z having not yet been subjected to detection 
processing is displayed on tiie ID panel and tiie IDX 
panel to clarify the existence thereof so that it is showily 
colored or patterned with a color different from colors 
indicative of frames subjected to detection processing 
or colors indicative of selected zones. 
[01 67] For example, on the ID panel, the frame por- 
tion 904 of the detection unprocessed zone 901 in tiie 
film image 902 is showily colored or patterned with a 
discriminative color or pattern or the combination 
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thereof, as shown in Fig. 24. On the IDX panel, the 
detection unprocessed zone 608 is showily colored or 
patterned with the different color or pattern or the com> 
tMnation thereof, as shown in Fig. 23 and mentioned 
above. In the shown example, the detection of the char- 
acter "X" is effectuated in all regions of the detection 
zone 607. In the case where the operator desires to 
make the character detection tor images in the detection 
unprocessed zone 901 . the operator doiriDte-dicks the 
detection unprocessed zone representation 608 on the 
IDX panel 6001 . At this time, ten leading frame images 
in the detection unprocessed zone 608 are displayed in 
the fim image 901 on the ID panel 5001. At the same 
time, a beam representation 10 is provided for the M- 
icon 103Z conesponding to the detection unprocessed 
zone 608. Thereby, the specif ication of an M-icon under 
edition and a relationship between that M-ioon and an 
editing work become dear. The beam representation 1 0 
is a beam-bike representation which makes the inter- 
connection of those data representations on separate 
windows which tiave a relation therebetween, as shewn 
in Fig. 20. The fomn. color, pattern, etc. of the t>eam rep- 
resentation 10 may be arbitrary. For example, the beam 
representation 10 can be displayed in a discriminative 
mode which changes depending on tiie amount of infor- 
mation or the contents of information. 
[0168] Though a sequence in accordance with 
which zones to be newly subjected to detection 
processing are added on the IDX panel follows the 
sequence of edited tree structures in the editing area, 
the sequence of addition may be the sequence of 
attachment or the sequence of display in the material 
area. 

[0169] Now. the description will be made of ttie 
case where a detection processing is actually per- 
formed. At this time, number information of M-icons hav- 
ing already been subjected to detection is stored in the 
memory 3020. Therefore, it is only required to make the 
detection for frames of an M-icon having not yet been 
subjected to detection processing. On the ID panel, the 
result of detection is distinctively displayed with a detec- 
tion arrow 506 (indicative of a begin point) arvj a detec- 
tion arrow 507 (indicative of an erxi point) the colors of 
which are different from those of detection arrows for 
the already perfornied detection processing. On the IDX 
panel, frames subjected to the present detection 
processing are darified so that they are showily pat- 
terned with a color different from that of frames having 
already been subjeded to detection. This representa- 
tion with different color is finished when the operator 
moves the detection arrow 506 or 507 or turns on the 
OK txjtton 541. When the tree structure is invoked 
again, such representation with different color turns to 
the representation having the same color as tiie detec- 
tion arrows for the already performed detection process- 
ing. 

[0170] Finally, these panels are dosed by turning 
on the OK button 541 or the CANCEL button 542 on the 



ID panel 500 to complete ttie editing work. 
[0171] The editing window, the ID panel and the 
IDX panel can be moved by use of tiie input device 3090 
such as a nrxxise. 

5 [0172] Figs. 25 and 26 are diagrams showing an 
embodiment of tiie editing window according to tiie 
present invention. Fig. 25 is substantially the same as 
Fig. 18. In Figs. 25 and 26. reference numerals 201. 202 
and 203 denote monitor display screens, numerals 50' 

10 and 50" ID panels, numerals 60' and 60" IDX panels, 
and numerals 100* and 100" editing windows. 
[0173] As shown in Fig. 25. the displayed editing 
window 100*. ID panel 50' and IDX panel 60* may be 
positioned so that one or all of the panels do not over- 

75 lap. The monitor 3080 may be construded so that it fias 
ttie plurality of display screens 201. 202 and 203 on 
which tiie ID panel 50". the editing window 100" and th 
IDX panel 60" are displayed, respedively. as shown in 
Fig. 26. Also, a plurality of monitors may be provided. In 

20 this case, a beam representation becomes possible, for 
example, by preliminarily informing the CPU 3010 of the 
installation positbns of the monitors 201 . 202 and 203. 
[0174] According to tiie foregoing embodiments, 
when the edition of a motion pidure is made using a 

25 motion picture editing apparatus, the contents of 
change made in an editing work are displayed on an ID 
panel and an IDX panel simultaneously. Therefore, it 
becomes possit^e to perform a work of confirming infor- 
mation of the respective panels (with the adual image 

30 data seen on the ID panel in units of one frame and 
seen on the IDX panel in units of one M-icon or frames 
corresponding thereto). As a result, it is possible to 
improve tiie effidency of edition. 
[0175] In the foregoing embodiments, there is the 

55 following problem. In order that after tiie detedion of a 
specified image such as a character or ttie like, a 
domain resulting in the effectuation of detection of the 
specified image is to be changed, it is required tiiat an 
operation of newly generating a t^egin point and an 

40 operation of deleting the original begin point should be 
performed or an operation of newly generating an end 
point and an operation of deleting the original end point 
shouki be performed. These operations are complex 
and are liat^le to make an error. 

45 [0176] Another problem concerns a change in 
objed frame. When an operator makes tiie change in 
position/size of an objed frame which lies in one frame 
image (hereirtafter referred to as frame simply), there 
results in that the klenticat objed frames in the succeed- 

so ing frames are also changed to the same position/size 
as the one okDjed frame, that is. the change is automat- 
ically made lor objed frames for which the operator 
does not intend to make the change. Therefore, in the 
case where an objed frame in one frame is changed, it 

55 is necessary to pay attention to objed frames in the suc- 
ceeding frames. 

[0177] In an embodiment of tiie present invention 
which will be described in the following, the above prob- 
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lem is overoome. Ihereby providing a correcting method 
in which the simplified operationality is attained, an error 
is hard to generate, and it is easy to find out an error 
even rf the error is generated. 

[0178] T solve the above problem, an Image s 
detecting method according to one emlxxJiment of the 
present invention enalsles the implementation of the 
change the position of a detection arrow and the change 
of a detection effectuating domain by interlocking the 
detection an-ow and the detection effectuating domain 10 
with each other and moving the detection arrow. In the 
case where a detection effectuating domain is to be 
newly added, there is implemented a construction in 
which it is possitile to perform an operation by clicking 
the ADD txjtton after the domain to be newly added is is 
designated on a film image beforehand. Similarly, in the 
case where a detection effectuating domain Is to be 
deleted or cleared, there is implemented a construction 
in which it is possit)le to perform an operation by clicking 
the CLEAR txitton after the domain to be deleted added 20 
is designated on a film Image beforehand, in the case 
where it Is desired to move the detection arrow to the 
nd of a film image being displayed so that it is further 
moved to a zone for which no film image is being dis- 
played, there is implemented a construction in which 2S 
frame images are automatically scrolled. 
[0179] According to an image detecting method 
according to an emkxxjiment of the present invention in 
the case where an object frame is changed, the change 
in object frame is made for only a selected portion on a 30 
film image. Further, object frames outside of a desig- 
nated area can individually be changed every frame. 
[0180] Thus, there is realized the image detecting 
method in which the simplified operationality is attained, 
an error is hard to generate, and it is easy to find out an 35 
error even if the error is generated. 
[0181] The method for change in detection effectu- 
ating domain and olDject frame according to the present 
invention will now t>e described. 

[0182] Rrst, the change in detection effectuating 4o 
domain (the enlargement/reduction of a domain and the 
joint of domains) by the dragging of a detection arrow 
will be described using Fig. 27 which shows a flow chart 
illustrating an embodiment of a processing performed in 
the case where the cfiange in detection effectuating 45 
domain is made. 

[0183] The case of the reduction of a domain will 
now be described in conjunction with a detection effec- 
tuating domain 21 1 exemplarily shown in Fig. 28. 
[0184] Rg. 28 is a diagram showing an embodiment so 
of an ID panel when the confirmation or change of the 
result of detection according to the present invention is 
made. Reference numeral 2000 denotes an ID panel, 
numerals 200. 201. 202 and 203 detection an-ows! 
numerals 204. 205. 206. 207 and 208 object frames, ss 
numeral 209 a film image, numerals 21 1 and 212 detec- 
tion effectuating domains, numerals 213. 214, 215. 216. 
217, 218, 219. 220, 221 and 222 frame images 



arranged in the film image 209 in order along the time 
axis from the left of the screen, numeral 223 a skip but- 
ton, numeral 224 a display box, numeral 231 an ADD 
button, numeral 232 a CLEAR button, numeral 305* an 
enlarged version of the fram image 213 positioned at 
the leftmost portion of the film image 209. numeral 306 
an enlargement display area where the enlarged image 
305* is displayed, numeral 301 an OK button, numeral 
302 a CANCEL button, and numeral 313 an OBJECT 
FRAME ADJUST button. 

[0185] The detection arrows 200 to 203 displayed 
on the upper side of the film image 209 are arrows indi- 
cating the Ijeginning and end of each detection effectu- 
ating domain. The detection arrow 200 or the detection 
arrow 202 is the head of detection called a begin point. 
The detection arrow 201 or the detection an-ow 203 Is 
the termination of detection called an end point. A 
domain sandwiched between the detection arrow 200 of 
the begin point and the detection arrow 201 of the end 
point is the detection effectuating domain 212, and a 
domain sandwiched between the detection arrow 202 of 
the begin point and the detection arrow 203 of the end 
point is the detection effectuating domain 21 1 . Each of 
the object frames 204 to 208 displayed in the frame 
images belonging to the detection effectuating domain 
21 1 and the detection effectuating domain 212 indicates 
a position at which a character "X" exists in the frame 
image. The object frame is displayed as the result of 
character detection. 

[0186] When the processing shown in Fig. 27 is 
started, tiie pointer 810 is moved to the position of the 
detection arrow 202 of the begin point in the detection 
effectuating domain 211 made of an object of change 
and is then clicked (step 2100). 
[0187] Next, the mouse pointer 810 is dragged to 
drag the detection aaow 202 into a desired right/left 
position (step 2101). 

[0188] Thereafter, the check is made of whether or 
not the mouse pointer 810 being dragged in step 2101 
reaches the right/left extreme of the film image 209 
(step 2102). When the mouse pointer 810 reaches the 
right/left extreme of the film image 209, the flow goes to 
step 2103. When tfie case is not so, the ftow goes to 
step 2104. 

[01 89] In step 21 03, tfie film image 209 is subjected 
to a scroll processing. For example, when the mouse 
pointer 810 being dragged reaches the right extreme of 
the film image 209. frame images positioned on the fur- 
ther right side of the film image 209 are slid leftwards so 
that they are displayed on the film image 209. Similarly, 
when tiie mouse pointer 810 being dragged reaches the 
left extreme of the film image 209. frame images posi- 
tioned on the further left side of the film image 209 are 
slid rightwards so that they are displayed on ttie film 
image 209. Then, ttie flow goes to step 2104. 
[01 90] The number of frames to be scrolled at once 
can be set in units of the predetermined number of 
frames (for example, in units of one or more frames). A 



20 

BNSDOCID; <EP 1033718A2_I_> 



39 



EP 1033 718 A2 



40 



mefthod of changing this unit of scroll or the number of 
fianies to be scrolled may include, for example, a 
method in which an upward/downward arrow on the skip 
button 223 is attached to increase or decrease the 
number of frames to t>e skipped which is displayed in 5 
the display t)OX 224 on the left side of the skip button 
223 (wherein the attachment of the upward arrow 
causes the increase in number of frames and the 
attachment of the downward arrow causes the 
decrease in number of frames), a method in which the 10 
number of frames is inputted into the display box 224 
directly from the keyt^ard. and a method in which the 
number of frames is entered into a configuration file anti 
that numeric value is written into the memory at the time 
of start of an application. The configuration file includes 75 
various setting conditions of the application (for exam- 
ple, the directory setting of a destination for output of the 
file) which are designated by the operator beforehand. 
[0191] Next, the judgement is n^ade of whether or 
not the dragged mouse pointer 8 1 0 reaches the position 20 
of the detection arrow 203 of the end point paired with 
the detection anrow 202 (step 2104). When the case is 
so. the flow goes to step 2106. When the case is not so. 
the flow goes to step 2105. 

[0192] In step 2105. the detection arrow 202 is 25 
moved in conpliance with the motion of the mouse 
pointer 810. Then, the flow goes to step 2106. 
[0193] In step 2106. the judgement is made of 
whether or not the dragging of the mouse pointer 810 is 
completed. If the dragging is not completed, the flow 30 
returns to step 2102 to repeat the processing. If the 
dragging is completed, the flow proceeds to step 2107. 
[0194] In step 2107, the flow branches off here to 
(1) to (5) (steps 2181 to 2185) in accordance with the 
stop position of the detection arrow 202. 35 

(1) Step 2181: In the case the begin point (or the 
detection arrow 202) overlaps the end point of the 
detection effectuating domain 21 1 (or the detection 
arrow 203), the detection effectuating domain 21 1 40 
IS deleted. Namely, data of the detection arrow 202 

(or the begin point), the detection arrow 203 (or the 
end point) and the object frames 206 to 208 in the 
corresponding domain is deleted from the data 
structure. Thereby, the display of the detection 45 
arrows 202 and 203 and the object frames 206 to 
208 in the corresponding domain is extinguished. 

(2) Step 21 82: In the case where the begin point (or 
the detection arrow 202) lies in the detection effec- 
tuating domain 21 1 . the extent of detection effectu- so 
ating domain 211 is reduced. For example, 
provided that the detection arrow 202 is dragged 
rightwards by two frames, data of the detection 
arrow 202 is shifted rightwards by two frames so 
that data of the object frames 206 and 207 is ss 
deleted from the data structure. Thereby, the detec- 
tion arrow 202 of the begin point is displayed on the 

left side of the frame 221 arxl the display of the 



object frames 206 and 207 is extinguished. 

(3) Step 1 283: in the case where the t>egin point (or 
the detection an-ow 202) lies between the detectran 
effectuating domains 211 and 212, the detection 
effectuating domain 211 is enlarged. For example, 
provided that the detection an-ow 202 is dragged 
leftwards by one frame, data of the detection arrow 
202 is shifted leftwards k>y one frame so that data of 
an ot>ject frame is newly added to the data struc- 
ture. This newly added object frame takes the same 
size/position as the object frame 206. Thereby, the 
detection arrow 202 is displayed on the right side of 
the frame image 21 7 and the new object frame hav- 
ing the same size/tx>sition as the object frame 206 
is displayed in the frame 218. 

(4) Step 1284: In tiie case where the begin point (or 
the detection arrow 202) lies in the detection effec- 
tuating domain 212 (inclusive of the case where it 
overlaps the detection arrow 200 or 201), two 
detection effectuating domains 211 and 212 are 
joined. Namely, data of the detection arrow 202 and 
the detection arrow 201 of the end point is deleted 
from the data structure and data of three object 
frames falling k>etween the domains 21 1 and 212 is 
newly added to the data structure. These newvly 
added object frames are generated through linear 
interpolation in position made between the object 
frames 206 and 205. 

(5) Step 1 285: In the case where the begin point (or 
the detection arrow 202) lies on the further left side 
than the detection effectuating domain 212. tiie 
detection effectuating domain 211 is enlarged so 
that it is joined with a plurality of domains on the left 
side thereof (inclusive of the domain 212). The 
detection arrows and object frames take the combi- 
nation of the case of joint and the case of enlarge- 
ment. 

[01 95] The object frame change based on the linear 
interpolation will now be described using Fig. 29. Fig. 29 
shows an ID panel showing an emkxxliment of the result 
of joint of the detection effectuating domains 21 1 and 
212. In Fig. 29. the same parts as those in Fig. 28 ar 
denoted by the same reference numerals as those used 
in Fig. 28. Numeral 2001 denotes an ID panel, numeral 
200* a detection arrow indicative of a begin point, 
numeral 203* a detection arrow indicative of an end 
point, numeral 209' a film image, numeral 211' a detec- 
tion effectuating domain, and numerals 251 to 253 
object frames. 

[0196] Fig. 29 shows a state resulting from the left- 
ward movement of the begin point of the detection effec- 
tuating domain 211 (or the detection arrow 202) for the 
change in domain to overlap the end point of the detec- 
tion effectuating domain 212 (or the detection anow 
201) (see Fig. 28). The detection arrows 201 and 202 
are deleted and the detection effectuating domains 21 1 
and 212 are jointed to form a detection effectuating 
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domain 211*. 

10197] The new object frames of the frame images 
216 to 218 newly added to the detection effectuating 
domain are determined from positions obtained by the 
equi-interval division of lines which connect the apexes s 
of the object frame 205 of the frame image 215 on the 
left side of the new object frames and the corresponding 
apexes of the object frame 206 of the frame image 219 
on the right side of the new object frames. Though the 
connecting lines are depicted in Fig. 29 as dotted lines w 
for the convenience of illustration, those lines are not 
actually displayed. These linearly interpolated object 
frames 251 to 253 are displayed. Thus, the frame 
images 216 to 218 are respectively allotted with the 
object frames 251 to 253 having their positions and is 
sizes wWch equally change from that of the object frame 
205 to that of the object frame 206. 
[0198] Though the change in object frame is made 
by clicking the OBJECT FRAME ADJUST button 313 for 
the switching to an object frame change mode, the 20 
details thereof wilt be mentioned later on. VS/hen the OK 
button 301 is clicked after the change in object frame, 
the ID panel 2001 is closed with the result of change 
held. On the other hand, when the CANCEL button 302 
is clicked, the ID panel 2001 is closed with the result of 2S 
change cancelled. In the case where the OBJECT 
FRAME ADJUST button 313 is clicked to switch the ID 
panel 2001 to the object frame change mode and the 
OBJECT FRAME ADJUST button 313 is clicked again, 
the ID panel 2001 returns to the former display mode. 30 
[0199] Next, the change in ot>ject frame will be 
described. The change in object frame divides into inter- 
polation change and individual change. The interpola- 
tion change is used when it is desired to correct a 
plurality of otjject frames at once (or en bloc). The indi- 35 
vidua! change is used wvhen object frames are to be 
changed one by one. 

[0200] First, the description using Figs. 30. 31 and 
32 will be made of an emtxxliment in the case where a 
plurality of object frames are interpolated at once. Fig. 4o 
30 is a flow chart showing an embodiment of a process- 
ing in the case where a plurality of object frames are 
interpolated at once. Figs. 31 and 32 are diagrams 
showing an emtjodiment of an ID panel in the case a 
plurality of object frames are interpolated at once. In 45 
Figs. 31 and 32. the same parts as those in Fig. 28 are 
denoted by the same reference numerals as those used 
in Rg. 28. Numerals 2004 and 2004' denote ID panels, 
numerals 475 and 475' film images, numeral 473 a 
detection arrow indicative of a begin point, numeral 474 so 
a detection arrow indicative of an end point, numerals 
453 to 462 frame images, numerals 463 to 470 and 464' 
to 467' object frames, numeral 471 a detection effectu- 
ating domain, and numeral 472 a selected frame area. 
[0201] When the processing shown in Fig. 30 is ss 
started on the ID panel shown in Fig. 31. the operator 
selects frame images 454 to 459 (that is, a frame area 
472) by use of the mouse pointer 810 (step 2150). 



[0202] Next, in the case where the OBJECT 
FRAME ADJUST button 313 is clicked (step 2151). the 
flow proceeds to step 2152. 

[0203] In step 2152. the object frame display mode 
is turned to a change mode. In the object frame change 
mode, eight square points are added to each of 
selected otsject frames 463 to 470. as shown in Fig. 31 . 
[0204] Next, whether or not the number of object 
frames selected in step 2150 is one or plural is judged 
to determine whether or not the change in object frame 
shoukl be made en bloc (step 2153). Namely, when plu- 
ral object frames are selected, the flow goes to step 
2154 in order to make the en bloc change of these 
frames. When only one object frame is selected, the 
ffow goes to step 2155 in order to make the change of 
only the one frame. 

[0205] In descrbing the enbloc change made in 
step 21 54, consider by way of example the case where 
the necessity of making a change in position/size of the 
ot)ject frames 463 and 468 is little whereas the posi- 
tion/size of the object frame 466 is most different from 
the pixel region of a character "X" appearing in the 
frame image 457. In this case, the object frame 466 is 
subjected to deformation or change. The object frame 
466 is deformed by dragging the mouse pointer 810. 
When the deformation of the object frame 466 is com- 
pleted (or the dragging is stopped), the object frames 
463 to 468 in the selected frame area 472 are subjected 
to interpolation change with the object frames 463 and 
466 taken as origins or reference frames and with the 
ot)ject frames 466 and 468 taken as reference frames, 
in a manner similar that described in conjunction with 
Fig. 29. An example of the changed object frames is 
oljject frames 464\ 465\ 466* and 467- displayed in Fig. 
32. Dotted lines depicted in Rg, 32 for the convenience 
of illustration are not actually displayed. 
[0206] In the case where only one object frame is 
selected (step 2153). only the selected object frame is 
subjected to change (step 21 56). 
[0207] When the change in step 2 1 54 or step 21 55 
is completed, the flow proceeds to step 2156 in which 
the judgement Is made of whether or not the OBJECT 
FRAME ADJUST button 313 is clicked again. If the but- 
ton 313 is not clicked, the flow returns to step 2153 to 
continue the change processing. If the button 313 is 
clicked, the flow proceeds to step 2157. 
[0208] In step 2157, the object frame display mode 
is turned to the former mode so that the change mode is 
cancelled. 

[0209] In the case where the OK button 301 is 
clicked, the ID panel 2004' is closed with the result of 
change held. On the other harxi. in the case where the 
CANCEL button 302 is attached, the ID panel 2004' is 
closed with the result of change cancelled. 
[0210] In the foregoing embodiment, one exanrple 
of the image detecting method has been described in 
conjunction with an example in which a character is 
detected. However, it is self-evident that an object 
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image other than characters can b detected by desig- 
nating the combination of specific colors. Further, it is 
needless to say that the present inv ntion can also be 
applied in conjunction with a detecting method other 
than that mentioned at>ove. 

[02111 Now, the procedure for storing data of the 
result of detection of a character always in the informa- 
tion region of an M-icon of the lowermost layer of a hier- 
archical structure will be described in reference to a flow 
chart shown in Fig. 33. In the at>ove-mentioned embod- 
iment of the present invention, it has been shown that in 
the case where an upper layer of tfie tree structure, tor 
example, the iy/l-icon 1220 "a" (Fig. 8) is attached to per< 
form a processing for the detection of the character "A", 
the data of the result of detection of the character "A" is 
stored into data structures corresponding to M-icons 
1210 "c" and "d" in the lowermost layer of the tree 
structure. The operation of a processing performed in 
this case will be described referring to the flow chart 
shown in Fig. 33 and data structures shown in Figs. 35 
to 41. 

[021 2] In step 8001 for scene number acquisition, a 
scene number in a data region 72 of a data structure 
7000(a) (see Fig. 35) of the attached M-icon 1220 "a" is 
acquired. Namely, when an M-icon corresponding to a 
scene is attached in order to detect the character '*A^ 
an attach structure of the attached M-icon is generated 
con-esponding to a scene number. For example, 
assume that an operator attaches the M-icons 1031 to 
1033 (Fig. 4) in a sequence from a smaller number to a 
larger number. In the case where the number of 
attached M-icons is thus not one but plural, a pointer to 
an attach structure 7000(b) (Fig. 35) of the next 
attached M-icon is acquired from a data region 71 of the 
attach structure 7000(a) and a scene number is 
acquired from a data region 74 of the attach structure 
7000(b). Then, a pointer to an attach structure 7000(c) 
of the next attached M-icon is acquired from a data 
region 73 of the attach structure 7000(b) and a scene 
number is acquired from a data region 76 of the attach 
structure 7000(c). Then, the address of the next attach 
structure is acquired from a data region 75 of the attach 
structure 7000(c). Thus, scene numbers are acquired 
by the number of attached M-icons. 
[0213] Next or in step 8002 for scene structure 
acquisition, there is acquired a scene structure which 
has the same scene number as the scene number 
acquired in step 8001. This scene structure is gener- 
ated as an editing work is performed to generate a tree 
structure in the editing area 1010. For example, in the 
case of the attach structure 7000(a) shown in Fig. 35, a 
scene structure 7100 shown in Fig. 36 is found out refer- 
ring to the scene number stored in the data region 72 of 
the attach structure 7000(a). In the case of the attach 
structures 7000(b) and 7000(c). the reference is made 
to the scene numbers stored in the data regions 74 and 
76 of the respective attach structures. 
[0214] Next or in step 8003 for display dass judge- 



ment, a display class is judged from a display dass 
number in the scene strudure. For example, in the case 
where the attached M-icon is the M-icon 1220 "a" which 
represents a scene of the tree structure 1040 having 
5 already been edited in the editing area 1 01 0 on the edit- 
ing window 1000 shown in Fig. 8. a display dass 
number stored in a data region 78 of a scene structure 
7100 corresponding to a scene number of that scene is 
acquired. Namely, data regions 78, 82, 86 and 90 are 
10 stored witti display class numbers or numeric values 
each of which indicates a layer in the tree structure for 
the M-icon corresponding to that scene structure. For 
exanple, the display class number equal to "0" indi- 
cates the lowermost or first layer, the display dass 
15 number equal to "1 " indicates the second layer, the dis- 
play class numk>er equal to "2" indicates the third layer, 
and so forth. Now, the data region 78 is stored with "1 " 
representing the layer of the M-icon 1220 "a" and the 
data regfons 82, 86 and 90 are stored with "0" repre- 
20 senting the layer of the M-icons 1 210 V. "c" and "d". In 
step 8003, the judgement is made of whether or not the 
scene structure is one of the lowermost layer. If it is the 
lowermost layer, the flow goes to step 8004. If it is not 
the lowermost layer, the flow goes to step 8008. The 
25 judgement of whether or not the scene structure is one 
of the lowermost layer, can be made by refen-ing to the 
data region in which the display dass number is stored. 
Namely, the flow goes to step 8004 if a numeric value 
stored in the data region for display class nun*er is "0" 
30 and step 8008 if it is equal to "1 " or larger than that. 
[0215] In step 8008 for lower layer scene structure 
acquisition, there is acquired a scene structure which 
lies in a layer lower than the tree structure layer of the 
scene structure acquired in step 8002. For example, 
35 provided that a scene structure corresponding to a 
scene number of the M-icon 1220 "a" shown in Fig. 8 is 
the scene structure 7100 (Fig. 36), a pointer to (or an 
address of) a scene structure 7100A corresponding to 
the M-icon 1210 *t)" at the leftmost position of the lower 
40 tree layer is stored in a data region 77 of the scene 
structure 7100. With the reference made to this pointer, 
the scene structure 7 100 A can be acquired. Further, the 
reference is made to a data region 81 of the scene 
structure 71 OOA. thereby making it possifcjie to acquire a 
45 scene structure 7100B corresponding to the M-icon 
1210 "c" on the right side of the M-icon 1210 "b". Simi- 
larly, the reference is made to a data region 85 of the 
scene structure 7100B, thereby making it possible to 
acquire a scene structure 7100C corresponding to the 
50 M-icon 1210 "d" on the right side of the M-icon 1210 "c". 
If the lowermost scene structure(s) are acquired, the 
ffow returns to step 8003. 

[0216] In step 8004 for cut structure acquisition, the 
reference to a cut number stored in any of the data 
55 region 84. 88. 92 of the scene structures 71 OOA. 7100B 
and 7100C of the lowermost layer (Fig. 36) is made to 
acquire a cut structure which is storing the same cut 
number as that cut number. In an example shown in Fig. 
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36, data regions 83. 87 and 91 of the scene structures 
7100A, 7100B and 7100C are stored with scene num- 
bers to which the respective scene structures belong. 
Similarly, data regions 84. 88 and 92 of the scene struc- 
tures 7100A. 7100B and 7100C are stored with cut , 
nurtibers belonging to th lower layer off the respective 
scene structures. The reference to a cut number data 
region of a scene structure having a display class 
number equal to "0" is made to acquire a cut structure 
having the same cut number as that cut number. For ^ 

xample, in the case of the scene structure 7100A, the 
cut structure 7200 (Fig. 37A) is acquired, and In the 
case of scene structure 7100B. a cut structure 7200A 
(Fig. 38) is acquired. 

[0217] Next or in step 8005 for object class struc- t 
ture acquisition, the reference is made to a data region 
94 of the cut structure 7200 (Fig. 37A) in which a pointer 
to object class structure is stored. Thereby, an address 
for an object class structure 7300P is acquired in which 
information of the character detection result is stored, s 
Namely, an object class structure for storing information 
of character search is acquired on the Isasis of a data 
region of the cut structure indicating a pointer to object 
class structure. In the case of the cut structure 7200A 
shown in Fig. 38, the reference to a data region 95 is a 
made to acqiire an address for an object class structure 
7300S. 

[0218] Next or in step 8006 for data storage, each 
data of the result of detection is stored into the corre- 
sponding region of the object dass structure 7300P and 3c 
the succeeding structures. In other cases, each data of 
the result of character detection is stored into the corre- 
sponding region of the object class structure 730OS as 
shown in Rg. 38 and the succeeding structures as the 
same manner. 35 
[021 9] Next or in step 8007 for cut process comple- 
tion judgement the judgement concerning one attached 
M-icon for cuts in Its lowermost layer is made as to 
whether or not the processing up to the data storage is 
completed. If the processing is not completed, the flow 4o 
returns to step 8005, If the processing is conpleted. the 
flow proceeds to step 8009. 

[0220] Next or in step 8009 for process conpletion 
judgement the judgement concerning all list structures 
71 00 A to 71 OOC (Fig. 36) for the scene structures of all 45 
the attached M-icons is made as to whether or not the 
processing up to the data storage is completed. If the 
processing is not completed, the ftow returns to step 
8002. If the processing is completed, the flow is fin- 
ished. 5^ 

[0221] Next, the procedure for distinctively display- 
ing a domain having already been subjected to charac- 
ter detection processing and a domain having not yet 
been subjected to character detection processing will 
be described referring to a flow chart shown in Fig. 34 ss 
and the data structures shown in Figs. 35 to 41 . 
[0222] In step 8101 for character search start, an 
operator attaches the M-icons 1 030 to 1 032 in the mate- 



rial area 1020 by use of the mouse pointer 810 and 
clicks the character detecting button 1051 in the edition 
function button group 1050. thereby starting a character 
search. 

5 [0223] In step 8102 for data acquisition, the 
processing in the flow chart shown in Rg. 33 is per- 
formed to acquire data of the cut structures 7200(a) to 
7200(c) (Figs. 39 to 41) corresponding to the M-icons 
1030 to 1032. 

10 [0224] In step 81 03 for cut structure selection, one 
of the cut structures 7200(a) to 7200(c) shown in Rgs. 
39 to 41 is selected. For example, the cut structure 
7200(a) shown in Fig. 39 is selected. 
[0225] In step 8104 for object dass structure selec- 
ts tion, there is selected one of ofc^ect class structures 
which are under the cut structure selected in step 8103, 
For exanple. an object dass structure 7300D is 
selected. 

[0226] In step 8105 for object name acquisition, an 
20 object name in an object name data region of the object 
dass structure selected in step 8104 is acquired. For 
example, for the object class structure 7300D. an object 
name "B" in a data region 95 thereof is acquired. 
[0227] In step 8106 for other object name acquisi- 
25 tion. one cut structure other than the cut structure 
selected in step 8103 is selected and objed names In all 
ot)jed class strudures under the selected other cut 
structure are acquired. For example, for all of object 
dass structures 7300E to 7300H under the cut strudure 
30 7200(b) shown in Fig. 40. object names in data regions 
96 to 99 thereof are acquired. 
[0228] In step 81 07 for objed name judgement the 
judgement is made of whether or not the same object 
name as the objed name "B" acquired in step 8105 is 
35 included In the objed names acquired in step 8106. If 
the same object same is not found out. the ftow goes to 
step 81 08. If the same object same is found out. thef low 
goes to step 8109. 

[0229] In step 8108 for display of detection unsub- 
40 jected zone, the zone 605 portion of the detedion result 
display column 238 concerning the charader "B" on the 
IDX panel 600 shown in Rg. 1 is displayed as a detec- 
tion unsubjeded zone 250 (for example, with the zone 
showily black-colored). Then, the flow proceeds to step 
45 8100. 

[0230] In step 8109 for cfisplay of detection sub- 
jected zone, that is. for example, in the case where there 
is an objed class structure having the object name "B". 
the color distindion between a detedion effeduating 
50 domain and a detection ineffectuating domain is made 
for the zone portion of the detection result display col- 
umn of Fig. 1 corresponding to the cut structure. Then, 
the flow proceeds to step 81 00. 
[0231] For example, provided that for all o^ect 
55 dass structure 73001 to 7300J under the cut strudure 
7200(c) shown in Rg. 41, objed names are acquired 
from data regions 100 to 102 thereof (step 8106). it is 
determined that the objed class strudure 7300J has 
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the same object name "B". Therefore, a detection effec- 
tuating domain and a detection ineffectuating domain 
are distinctively displayed in the zone 606 portion of the 
detection result display column 238 concerning the 
character on the IDX panel 600 shown in Fig. 1 . 
[0232] In step 81 10 for completion 1 judgement, the 
judgement is made of whether or not the processings of 
steps 8106 to 8109 are completed for all the cut struc- 
tures. If the case is so, the flow proceeds to step 81 1 1 . 
If the case is not so. the flow returns to step 8106. 
[0233] In step 81 1 1 for completion 2 judgement, the 
judgement is made of whether or not the processings of 
steps 8103 to 81 10 are completed for all the cut struc- 
tures. If the case is not so, the flow retums to step 8103. 
If the case is so. the processing along the flow chart is 
completed. 

[0234] Next, the description will be descrbed of the 
data structures of Rgs. 35 to 41 used in the present 
invention, in the case where a processing for detection 
of a specified character is performed for frame images 
in one cut. one object class structure is generated tor 
one character. For example, the zones 604 of the detec- 
tion result display columns 237 and 238 on the IDX 
panel shown in Fig. 1 correspond to the object dass 
structure 7300P shown in Fig. 37A and the object dass 
structure 7300Q shown in Fig. 378. 
[0235] By virtue of "POINTER TO NEXT OBJECT 
CLASS STRUCTURE", a list structure for object dass 
structures can be formed. For example, the zone 604 of 
the detection result display column 237 on the IDX 
panel shown in Fig. 1 shows an object class structure 
7300P shown in Fig. 37A and a pointer to next object 
class structure field 120 in the object class structure 
7300P indicates an object class structure 7300Q corre- 
sponding to the zone 604 of the detection result display 
column 238. 

[0236] Provided that the zone 604 of the detection 
result display column 237 on the IDX panel shown in 
Fig. 1 is the zero-th zone, "0" is stored in a display dass 
number field 121 of the object class structure 7300P 
shown in Fig. 37A. Also, since the zone 605 of the 
detection result display column 237 is the 1-st zone. "1" 
is stored in a display class nunriber field 130 of the 
object class strudure 7300Q shown in Fig. 37B. 
[0237] By virtue of a pointer to object instance 
structure 1 22, it is possible to refer to an object instance 
structure in which data of one detection effectuating 
domain is stored. The details of the objed instance 
structure will be described later on. 
[0238] '^ATTACH FLAG" is a flag indicating whether 
or not a target image on the IDX panel (shown in. for 
example. Fig. 1) is clicked by the mouse. "ATTACH 
FLAG" = "1" indicates the presence of attachment and 
"ATTACH FLAG" = "0" indicates the absence of attach- 
ment. For example, in the case where the target image 
631 is clicked or attached by the mouse, an attach flag 
123 of the object class structure 7300P con^esponding 
to the zone 604 at the head of the detection result dis- 



play cdumn 237 is turned to "1 
[0239] "OBJECT NAME" is a name of a character 
made an object of character detection processing. For 
example, the charader name 634 "A" on the IDX panel 
5 shown in Fig. 1 is stored in an object name 124 of the 
objed class strudure 7300P. This name is a name 
inputted by the user as the object name 547 on the ID 
panel shown in Fig. 17. 

[0240] "NUMBER OF COLOR DESIGNATING 
10 FRAMES" indicates the number of color designating 
frames designated for setting a color pattern. For exam- 
ple, since three color designating frames 503. 504 and 
505 are set in the enlarged image 502 on the ID panel 
shown in Rg. 17, a numeric value in a number of color 
15 designating frames field 125 of the objed dass struc- 
ture 7300P is "3". 

[0241] A "COORDINATE LIST OF FOUR COR- 
NERS OF COLOR DESIGNATING FRAMES" (two- 
dimensional array) field stores the x- arKl y-coordinates 

20 of four corners of color designating frames on the 
screen. The x- and y-coordinates are the coordinates on 
the screen in the lateral and longitudinal directions, 
respectively. Coordinate sets exist by the number of 
color designating frames. For example, since three color 

25 designating frames are set in the enlarged image 502 
on the ID panel shown in Rg. 17, three coordinate sets 
each having the x- and y-coordinates of four corners of 
one cdor designating frame are stored in a coordinate 
list of four corners of color designating frames field 126 

30 of the objed class strudure 7300P. 

[0242] A "LIST OF COLOR DATA IN COLOR DES- 
IGNATING FRAMES" (two-dimensional an^ay) field 
stores data of designation colors in color designating 
frames. Color data sets exist by the nunnt)er of color 

35 designating frames. For example, since three color des- 
ignating frames are set in the enlarged image 502 on 
the ID panel shown in Fig. 17. three color data sets are 
stored in a list of cdor data in color designating frames 
field 127 of the object dass strudure 7300P. 

40 [0243] A "POINTER TO OBJECT INFORMATION 
STRUCTURE" field 128 of the object dass strudure 
7300P stores information for referring to an objed infor- 
mation structure in which information assodated with 
the charader made the objed of character detection 

45 processing (for example, l>irth date, copyright arxJ so 
forth) is stored. 

[0244] In the case where a combine processing is 
performed on the IDX pane! shown in Fig. 1. display 
dass numbers in objed class strudures in the same cut 

50 made an objed of combination are stored in a "COM- 
BINE CLASS LIST" (one<limensional array) field. For 
example, in the case where the detection result display 
columns 237 and 238 are combined, a numeric value in 
the display class number 121 of the objed class struc- 

55 ture 7300P and a numeric value in the display dass 
number 130 of the objed dass structure 7300Q are 
stored in the combine class list field 129 of the object 
dass strudure 7300P 
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[0245] Next, the object Instance structure will be 
described. In the case where a processing for detection 
of a specified character is performed for frame images 
in one cut. one object instance structure is generated for 
a domain in which one character exists. For example. 5 
the detection effectuating domains 614 in the zones 604 
of the detection result display column 237 on the IDX 
panel shown in Fig. 1 correspond to object instance 
structures 7400R 7400Q and 7400R shown in Figs. 
37A, 37B and 37C. 

[0246] By virtue of "POINTER TO NEXT OBJECT 
INSTANCE STRUCTURE", a list structure for object 
instance structures can be formed. For exanple. the 
first one of the detectfon effectuating domains 61 4 in the 
zone 604 of the detection result display column 237 on is 
the IDX panel shown in Rg. 1 is the object instance 
structure 7400P (Rg. 37A) and a pointer to next object 
instance structure field 140 in the object instance struc- 
ture 7400P indicates the object instance structure 
7400Q (Fig. 37B) con^esponding to the second one of so 
the detection effectuating domains 614. 
[0247] "DETECTION EFFECTUATING DOMAIN 
NUMBER" is a unique serial number for object instance 
structure. For example, provided that the first one of the 
detection effectuating domains 614 in the zone 604 of 2S 
the detection result display column 237 on the IDX 
panel shown in Fig. 1 is the zero-th domain. "0" is stored 
in a detection effectuating domain number 141 of the 
ot)ject instance structure 7400P shown in Fig. 37A. 
Since the second detection effectuating domain 614 is 30 
the 1-st domain. "1" is stored is stored in a detection 
effectuating domain number 161 of the object instance 
structure 7400Q shown in Rg. 378. 
[0248] A -DISPLAY CLASS NUMBER" field of the 
otjject instance structure stores a display class nunt>er 35 
of an object class structure to which that object instance 
structure belongs. For example, a numeric value in the 
display class number field 121 of the object class struc- 
ture 7300P is shown in a display dass number field 1 42 
of the object instance structure 7400P shown in Fig 40 
37A. 

[0249] "DETECTION EFFECTUATING DOMAIN 
BEGIN POINT* is a frame number indicating the begin- 
ning of a detection effectuating domain. For example, a 
frame number of the tjeginning of the first one of the 45 
detection effectuating domains 614 in the zone 604 of 
the detection result display colunwi 237 on the IDX 
panel shown in Fig. 1 is stored in a detection effectuat- 
ing domain begin point 1 43 of the object instance struc- 
ture 7400P shown in Fig. 37A. 50 
[0250] "DETECTION EFFECTUATING DOMAIN 
END POINT" is a frame number indicating the end of a 
detection effectuating domain. For example, a frame 
number of the end of the first one of the detection effec- 
tuating domains 614 in the zone 604 of the detection 55 
result display column 237 on the IDX panel shown in 
Fig. 1 is stored in a detection effectuating domain end 
point 144 of the object instance structure 7400P shown 



in Fig. 37A. 

[0251 ] "ATTACH FLAG" is a flag indicating whether 
or not a d tection effectuating domain on the IDX panel 
is clicked by the mouse. "ATTACH FLAG" = "1 " indicates 
the pr sence of attachment and "ATTACH FLAG" = "0" 
indicates the absence of attachment. For exanple. in 
the case where the first one of the detection effectuating 
domains 614 in the zone 604 of the detection result dis- 
play column 237 on the IDX panel shown in Fig. 1 is 
clicked or attached by the mouse, an attach flag 145 of 
the object instance structure 7400P shown in Fig. 37A is 
tumedto"1". 

[0252] A "POINTER 7D OBJECT POSITION 
STRUCTURE" field (146) of the object instance struc- 
ture stores information for referring to an object position 
structure in which data of an object frame indicating the 
position of a character made an object of character 
detection processing in one frame is stored. 
[0253] Next, the object position structure will be 
described. The object position structure stores data of 
an object frame indicating the position of a character 
made an object of character detection processing in one 
frame. For example, the object fiames 561 to 563 on the 
ID panel shown in Fig. 17 conrespond to object position 
structures 7500P to 7500R shown in Rgs. 37A to 37C. 
[0254] By virtue of "POINTER TO NEXT OBJECT 
POSITION STRUCTURE" (147). a list structure for 
ot)ject position structures can be fonned. 
[0255] For example, ttie object frame 561 on the ID 
panel shown in Fig. 17 is tfie object position structure 
7500P (Fig. 37A) and a pointer to next deject position 
structure field 1 47 of the object position structure 7500P 
indicates the object position structure 7500Q (Fig. 37B) 
corresponding to the object frame 562. 
[0256] "FRAME NUMBER" is a unique frame 
number of a cut in which an object frame exists. For 
example, a frame number of the object frame 561 on the 
ID panel shown in Fig. 17 is stored in a frame number 
field 148 of the object position structure 7500P shown in 
Fig. 37A. 

[0257] "DIMENSION OF OBJECT FRAME IN X- 
DIRECTION" indicates the length of an object frame on 
the screen in a lateral direction. For example, the lateral 
length of the object frame 561 on the ID panel showm in 
Fig. 17 is stored in a dimension of object frame in x- 
direction field 149 of the ot>ject position structure 7500P 
shown in Rg. 37A. 

[0258] "DIMENSION OF OBJECT FRAME IN Y- 
DIRECTION" indicates the length of an object frame on 
the screen in a longitudinal direction. For example, the 
longitiJdinai length of the object frame 561 on the ID 
panel shown in Fig. 1 7 is stored in a dimension of object 
frame in y<lirection field 1 50 of the object position struc- 
ture 7500P shown in Fig. 37A. 

[0259] "X-COORDINATE OF OBJECT FRAME" 
indicates the x-coordinate of tfie left/upper corner of an 
object frame on the screen. For example, tfie x-coordi- 
nate of the left/upper corner of the object frame 561 on 



26 



BNSDOCID: <EP 103371 SA2_I_> 



51 



EP1 033 718 A2 



52 



the ID panel shown in Fig. 17 is stored in an x-coorcli> 
nate of object frame field 151 of the object position 
structure 7500P shown in Fig. 37A. 
[0260] '^COORDINATE OF OBJECT FRAME" 
indicates the y-coordinate of the left/upper corner of an 
object frame on the screen. For example, the y-coordi- 
nate of the left/upper corner of the object frame 561 on 
the ID panel shown in Fig. 17 is stored in a y-coordinate 
of object frame field 152 of the object position structure 
7500P shown in Rg. 37A. 

[0261 ] According to the embodiment of the present 
invention mentioned above, the scroll processing for a 
film image by the dragging of a detection arrow or the 
attachment of the film image is possttile. Therefore, it is 
possible to make the adjustment of a long film image 
without operating a scroll bar. 

[0262] A second effect of the embodiment of the 
present invention is that it is possible to change only 
designated object frames regularly or systematically 
through interpolation change. 

Claims 



10 



IS 



features stored in a storage device (3060). and said 
representative Images to which said image detec- 
tion processing has not t>een applied are applied to 
said image detection processing on the basis of a 
same features. 

4. An image information displaying method according 
to Claim 1. wherein said predetermined ot^ect is 
the image of an object appearing in said motion pic- 
ture. 

5. An image information displaying method according 
to Claim 1. further comprising a step of deleting a 
part of said first information on said screen. 



6. An image information displaying method according 
to Claim 1. further comprising a step (Rg. 11) of 
performing separately plural image detection 
processings on the basis of different features for 
so predetermined ot^jects. and displaying in combina- 
tion of results of said plural image detection 
processings on said screen. 



1. An image information display method for editing a 
motion picture, wherein a representative image 25 
(1030-1033) representing a series of frame images 
forming the motion picture is displayed on a screen 
of a display device (3080) and a hierarchical struc- 
ture (1210, 1220. 1230. 1240) based on a plurality 
of the representative Images is displayed on said 30 
screen, said method comprising the steps of; 

selecting first representative images in each of 
which a predetermined object to be detected is 
included, from said representative images 3S 
based on an image detection processing; 
displaying a first information (614. 616) relating 
to said representative images including said 
first representative images, to which said 
image detection processing has been per- 4o 
formed, on said screen; and 
displaying a second information (250. 251. 
252) relating to said representative images, to 
which said image detection processing has not 
been performed, yet. with said first information. 45 
simultaneously, on said screen. 

2- An image information displaying method according 
to claim 1 , wherein in the step of displaying said first 
information, said first representative images includ- so 
ing said predetermined object and said representa- 
tive images not including said predetermined object 
are distinctively displayed. 

3. An image information displaying method according ss 
to Claim 1 , wherein said image detection process- 
ing is performed on the basis of one or more fea- 
tures possessed by said predetermined object, said 



7. An image information displaying method according 
to Claim 6, further comprising a step of dividing the 
combined result into the respective results. 

8. An image information displaying method according 
to Claim 1, further comprising a step of displaying, 
on said screen, said representative image which 
contains said predetermined object. 

9- An image information displaying method according 
to Claim 1. further comprising a step of displaying, 
said representative image combined with additional 
information which relates to said predetermined 
object detected by said innage detection process- 
ing, on said saeen. 

10. An image information displaying method according 
to Claim 1. further comprising a step of varying a 
display area of said first information on said screen. 

11. An image information displaying metiiod according 
to Claim 1, further comprising a step of selecting 
one of said representative images constructing said 
hierarchical structure and a step of applying said 
image detection processing to a series of frame 
images corresponding to said selected representa- 
tive image. 

12. An image information displaying method according 
to Claim 1, wherein said representative image is 
displayed in the form of a solid figure on said 
screen, and frame images containing said predeter- 
mined object detected in said image detection 
processing and frame images containing no said 
predetermined image are distinctively displayed on 
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a side face portion of said solid figure. 

13. An Image information displaying method according 
to Qaim 1. further comprising a step of displaying 
three windows simurtaneously on said saeen. said s 
three windows including a first window (1000) 
which displays said hierarchical structure to edit the 
motion picture, a second window (500) which dis- 
plays a plurality of frame images applied said image 
detection processing, and a third window which dis- io 
plays the detection result of said image detection 
processing, wherein the operations off said three 
windows are linked with each other. 

14. An image information displaying method according is 
to Qaim 13, wherein said representative images 
are displayed in said first window (1000) with an 
image size with which said hierarchical structure 
based on said representative images can be dis- 
played in said first window, and the displayed repre- 20 
sentative image is designated by the GUI so that 
the designated representative image is applied to 
said image detection processing. 

15. An image information displaying method according 25 
to Qaim 14, wherein a plurality of representative 
images as editing materials are displayed In said 
first window (1000) so that information of the result 

of detection displayed in said third window (600) 
and a representative image as the editing material 30 
related to said information of the result of detection 
are associated with each other (10). (Fig. 20) 

16. An image information displaying method according 

to aaim 1 . further comprising a step of changing at 35 
least one of said representative image from which 
said image detection processing is started to apply 
and said image detection processing is terminated 
to apply, the change being made by operating icons 
displayed on said screen. 40 

17. An image information displaying method according 
to Claim 1, wherein said image detection process- 
ing includes: 



displayed on said screen. 



45 



a step of displaying an object frame (308; 503- 
505) on the detected predetermined object 
(305: 520). said object frame designating a part 
of a selected frame image which contains said 
predetermined object; so 
a step of making the judgement of whether or 
not the same image information as image infor- 
mation off a region enclosed by said object 
frame is included in the plurality of frame 
images applied to said image detection ss 
processing; and 

a step of changing at least one of the size and 
position of said otjject frame by operating icons 



18. An image information displaying apparatus for edit- 
ing a motion picture, comprising: 

a display device (3080) having a screen for dis- 
playing representative images (1030-1033) 
representing a series of frame images forming 
the motion picture; 

a control device (3010) for controlling image 
information displayed on said screen; 
a storage device (3060, 3020) for storing said 
frame images, said image information and data 
for managing said frame images and said 
image information; and 

an input device for inputting data into said con- 
trol devk;e. wherein said conti-ol device 
includes means for constructing a hierarchical 
structure comprising a plurality of representa- 
tive images on the t>asis of data inputted by 
said input device; detecting means (309) for 
detecting a plurality of frame images including 
a predetermined object to be detected on the 
basis of an image detection processing, means 
for displaying a first information indicative of the 
detected frame image having said predeter- 
mined object on said screen, and means for 
displaying a secorxf information concerning 
frame images being not applied said image 
detection processing, on said screen. 

19. A computer program embodied on a computer 
readable medium for the display of image informa- 
tion for editing a motion picture, said computer 
readable medium having computer readable pro- 
gram code means comprising: 

means for displaying representative images 
(1030-1033) representing a series of frame 
images forming the motion picture on a screen 
of a display device (3080); 
means for constructing a hierarchical structure 
based on a plurality of representative images 
generated and displaying on said screen on tiie 
basis of inputted instruction data; 
means for detecting a plurality of frame images 
including in said representative image by 
applying an image detection processing, said 
frame images having a predetermined object to 
be detected; 

means for displaying a first information irKlica- 
tive of the detected frame image having the 
predetermined object on said screen; and 
means for displaying a second information 
(250-252) images being not applied witii said 
image detecting processing. 
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